RabbinatixDocs
Sign inGet started

Run and observe

Health checks

Tell Rabbinatix how to see whether a service is up in each environment. It calls each environment’s health URL while someone has the Connections view open, and shows the service as operational, degraded, down or in maintenance there.

Open Systems in the app

What a health check is

A health check tells Rabbinatix how to see whether a service is up. Each environment (Development, Production and your own) has its own check, with its own URL and settings. Rabbinatix calls that URL and turns the answer into a status for that environment:

StatusMeaning
OperationalThe service answered as expected
DegradedIt answered, but a rule says it is only partly working
MaintenanceA rule says it is down on purpose
DownIt didn't answer in time, couldn't be reached, or answered wrongly

Health always shows per environment: one dot per environment that has a check (hollow when the check is off) on the systems list and the system's header. An environment with tenants (see Tenants, below) has one dot, the worst of its tenants, and hovering it lists them. On Deployments, each environment's cell shows its dot. The system map shows only the environment picked in the header, Development included. In the Connections view a system with tenants in that environment has one part of its ring per tenant.

Set up a check

  1. Open the system and go to Running → Health checks. At the top it shows the system as a circle, as on the map, ringed by the status of each environment it checks. Below, it lists every environment.
  2. Click Set up next to the environment. To start from another environment's check, click Copy from that environment: everything is copied except the header values.
  3. Pick the method (GET, HEAD or POST) and type the Health check URL, such as https://api.example.com/health.
  4. Pick how often under Check every and how long to wait under Give up after. How often a check may run depends on the plan: Free every hour, Standard every 30 minutes, Premium every 15 minutes, Enterprise every 5 minutes.
  5. Under Operational when, set the expected HTTP status (2xx, 200, 200,204 or 200-299) and, if you like, text the response must contain.
  6. Click Test to try it once. The Test result shows the status, the reason and the start of the response. Nothing is saved.
  7. Click Save.

The switch next to each environment turns its check on or off at once, without losing it. Remove check deletes the check of the environment you are looking at; the other environments keep theirs.

Rules

Without rules a service is either Operational or Down. Rules let it report Degraded or Maintenance. Add one with Rule under Status from the response.

Rules are tried from top to bottom, before the Operational when test. The first rule that matches sets the status. Use the arrows to reorder them.

ConditionMatches when
JSON field equalsThe field at a path (status, checks.db.status, components[0].state) equals the value
Response JSON equalsThe whole JSON response equals the value
Response containsThe response text contains the value
HTTP status isThe status matches, such as 503, 5xx or 500-599

For example, if your service returns {"status": "degraded"} when its cache is down, add: if JSON field equals status = degraded then Degraded. Test highlights the rule that matched.

Headers and secrets

Add a Header when the health URL needs a key, such as Authorization: Bearer …. Each environment keeps its own header values, so Production's key never goes to Development. They are kept on the server, where the checks run, and are never shown again. The field then says it is saved. Leave it as is to keep the saved value, or type a new one to replace it.

For POST, you can also send a body. A body that looks like JSON is sent as JSON.

One check per environment

Each environment's check is complete on its own: URL, method, headers, schedule, expected answer and rules. Click an environment on Running → Health checks to see its check, its Service status and its history. Removing an environment removes its checks and their results.

Tenants

Some services run as several copies in one environment, such as one per country: https://it.example.com/health for Italy and https://uk.example.com/health for the UK. Give that environment tenants, and each tenant gets a full check of its own: URL, method, headers and their values, schedule, expected answer and rules.

  1. On the system's Running → Health checks, click the environment, then Add tenant at the top of its check.
  2. Name the current check (such as IT). It becomes the first tenant and keeps its results, history and header values.
  3. Name the new tenant (such as UK), type its health URL and click Add tenant. It runs at once. Set the rest in its form, or start it with Copy from another check (everything except header values).

The environment then lists its tenants, each with its status and its own switch. Click one to see its check, Service status, recent checks and uptime. Check now checks the tenant in view, and Check all now checks every tenant of the environment. Rename changes a tenant's name and keeps its history. Remove tenant stops checking it and keeps its past results. An environment has at most 25 tenants.

Each tenant is told about on its own: "Orders (Production · UK) is down", and its own Down ticket. The Connections view runs the due checks of every tenant of the environment picked in the header, and never another environment's.

When checks run

Checks run on the Rabbinatix server, not in your browser. They don't run on their own: they run when someone is looking.

  • While the Connections view is open. Open the system map and pick Connections. While it is open, Rabbinatix runs the checks of the environment picked in the header, and only those. Each check runs when its Check every time has passed, and again each time it comes round while the view stays open. A check set to run more often than the plan allows runs as often as the plan allows. Several people with the view open don't make a check run more often. The legend says when the last check ran.
  • Check now on the Service status card runs the check in view at once (the environment's, or one tenant's), even when the check is off.
  • When you save or turn on a check, it runs at once.

Nobody looking means nothing is checked. A service that goes down at night is noticed the next time someone opens the Connections view or clicks Check now.

The server doesn't call private or internal network addresses. A URL that starts with / is resolved against the app's address.

Every check in one place

Open Settings, then Health checks. The page lists every check, one row per service and environment (and tenant), with its URL, interval and status. It starts on the environment picked in the header; choose All environments to see every one. Set up opens a system's Health checks tab, and a button turns on every check that has a URL. They run at once.

Status, uptime and history

The Service status card shows the latest status, its reason, the HTTP status, the time it took, and a strip of recent checks. Recent checks lists the results one by one. The newest 60 results are kept per environment.

Uptime, last 30 days shows one bar per day: green at 99% or more, amber at 90% or more, red below. It counts only the checks that ran, so days nobody looked are grey.

Notifications and tickets

When a check finds that a service changed from one status to another in an environment, people are notified, such as "Orders (Production) is down", or "Orders (Production · UK) is down" for a tenant, with a link to that check on its Health checks tab. Each person can turn this off in their notification preferences under Service health changes (down, degraded, back up).

Both come from checks, so they come when a check runs (see When checks run, above). While an environment's latest check says Down, an urgent ticket "Down: <system> (<environment>)" stays open in New requests, one per tenant for an environment with tenants. It resolves by itself when a check passes and reopens if the service goes down again. Its Open health action opens that environment (and tenant) on the Health checks tab. Turn the rule off under Settings, Tickets, rule Services down. See Ticket automation.