> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# API Checks

> Ask your own backend whether a call did what it should

An API check is a pass/fail metric your own endpoint answers. After each call is analysed, Roark sends a request about the call to your API, and your API replies with `{ "passed": true | false, "reason": "..." }`. It runs on live calls and simulations alike.

Every other metric reads the conversation. An API check reads your system of record instead. An agent that says "your PIN has been changed" while the tool call behind it failed passes every transcript judge, but fails an API check that asks your card service whether the PIN actually changed.

Use an API check when the outcome lives in your backend:

* **State changes:** the PIN, address, or plan was actually updated.
* **Records created:** a booking, ticket, refund, or callback exists, with the right details.
* **Side effects:** an SMS or email went out, a payment was captured, a case was escalated.

***

## How It Works

**1. The call ends and is analysed**

API checks run with the rest of the call's metrics, from a [collector](https://docs.roark.ai/documentation/metrics/metric-collectors.md) on live calls or from a [run plan](https://docs.roark.ai/documentation/simulation-testing/run-plans.md) on simulations.

**2. Roark sends your request**

Roark fills your URL, headers, and body with data about the call and sends it. With no body, Roark POSTs the [default payload](#the-request-payload).

**3. Your endpoint answers**

Your endpoint replies `200` with `{ "passed": true | false, "reason": "..." }`. Roark records the result as Pass or Fail, with your reason shown next to it.

API checks are free: they don't use Roark's evaluation models, so they aren't billed like LLM Judge metrics.

***

## Creating an API Check

### In Studio

Open [Studio](https://docs.roark.ai/documentation/metrics/studio.md) in Author mode and pick the **API check** engine. The engine is locked after the first save.

![Studio Author mode with the API check engine selected: a POST request to https://api.example.com/roark/checks/pin-changed with a Test button beside the URL, a passed test result labelled PIN changed with HTTP 200 and the endpoint's reason, an Authorization header, a custom JSON body using \{\{call.properties.cardId}}, \{\{simulation.variables.newPin}} and \{\{call.id}} placeholders, and the Pass and Fail labels](https://mintcdn.com/roark/7qfPEfy6XsoMjlt3/images/metrics/api-check.png?fit=max\&auto=format\&n=7qfPEfy6XsoMjlt3\&q=85\&s=a9ece165cad16a7e30db27cf8f89f320)

**1. Request**

Choose `POST` or `GET`, enter your endpoint URL, and add any headers your API needs, such as an `Authorization` header. Type `{{` in the URL, a header, or the body to insert a [placeholder](#placeholders).

**2. Body**

Leave the body as it is to send the default payload, or edit it into the JSON your endpoint expects. Turn on **Preview sent body** to see the body with its placeholders filled in for an example simulated call.

**3. Test**

Click **Test** next to the URL to send the request once, filled from the example payload. Studio shows the result (Pass, Fail, Error, or Not sent), the HTTP status, your reason, the body it sent, and your endpoint's response. Nothing is saved, so you can test before the metric exists.

**4. Labels**

Name the two outcomes, for example **PIN changed** and **PIN unchanged**. They default to **Pass** and **Fail**.

> [!TIP]
>
> The test fills run variables, call properties, and persona fields the example call doesn't have with `test-<name>`, so a placeholder like `{{simulation.variables.newPin}}` sends `test-newPin`. Have your endpoint accept those values while you set it up, or check the sent body to confirm each value lands where you expect.

### With the SDK or API

Create the metric with `calculationType: "HTTP_REQUEST"`:

```typescript Node theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
import Roark from '@roarkanalytics/sdk'

const client = new Roark({ bearerToken: process.env.ROARK_API_BEARER_TOKEN })

const metric = await client.metric.createDefinition({
  calculationType: 'HTTP_REQUEST',
  name: 'PIN changed',
  httpRequest: {
    url: 'https://api.example.com/roark/checks/pin-changed',
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.CARD_SERVICE_TOKEN}` },
    body: '{"cardId": "{{call.properties.cardId}}", "expectedPin": "{{simulation.variables.newPin}}"}',
  },
  booleanTrueLabel: 'PIN changed',
  booleanFalseLabel: 'PIN unchanged',
})

console.log(metric.data.slug) // pin_changed
```

```python Python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
import os
from roark_analytics import Roark

client = Roark(bearer_token=os.environ.get("ROARK_API_BEARER_TOKEN"))

metric = client.metric.create_definition(
    calculation_type="HTTP_REQUEST",
    name="PIN changed",
    http_request={
        "url": "https://api.example.com/roark/checks/pin-changed",
        "method": "POST",
        "headers": {"Authorization": f"Bearer {os.environ['CARD_SERVICE_TOKEN']}"},
        "body": '{"cardId": "{{call.properties.cardId}}", "expectedPin": "{{simulation.variables.newPin}}"}',
    },
    boolean_true_label="PIN changed",
    boolean_false_label="PIN unchanged",
)

print(metric.data.slug)  # pin_changed
```

```bash cURL theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
curl -X POST https://api.roark.ai/v1/metric/definitions \
  -H "Authorization: Bearer $ROARK_API_BEARER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "calculationType": "HTTP_REQUEST",
    "name": "PIN changed",
    "httpRequest": {
      "url": "https://api.example.com/roark/checks/pin-changed",
      "method": "POST",
      "headers": { "Authorization": "Bearer sk_live_..." },
      "body": "{\"cardId\": \"{{call.properties.cardId}}\", \"expectedPin\": \"{{simulation.variables.newPin}}\"}"
    },
    "booleanTrueLabel": "PIN changed",
    "booleanFalseLabel": "PIN unchanged"
  }'
```

The response is the metric definition, with `type: "BOOLEAN"` and your `httpRequest` echoed back.

| Field | Type | Required | Description |
| :- | :- | :- | :- |
| `httpRequest.url` | string | Yes | Public `http(s)` URL, up to 2048 characters. May contain placeholders, which are URL-encoded |
| `httpRequest.method` | string | No | `POST` (default) or `GET`. A `GET` request has no body |
| `httpRequest.headers` | object | No | Up to 20 headers. Values may contain placeholders |
| `httpRequest.body` | string | No | A JSON template, up to 20,000 characters. Omit it, or pass `null`, to send the default payload |
| `booleanTrueLabel` / `booleanFalseLabel` | string | No | What a passed and a failed check are called. Default `Pass` and `Fail` |
| `analysisPackageId` | string | No | Package to add the metric to. Defaults to your project's Custom Metrics package |

To change the request, `PUT /v1/metric/definitions/{idOrSlug}` with the whole `httpRequest`. Headers you leave out are removed, and leaving out `body` switches back to the default payload. Each change creates a new metric version, so earlier results keep the request they ran with.

The API answers `400` when:

* the URL is not public `http(s)` (localhost, private, or internal addresses are refused)
* a placeholder doesn't exist in the payload
* the body isn't valid JSON once its placeholders are filled in
* a `GET` request has a body, or there are more than 20 headers

> [!NOTE]
>
> API checks aren't part of [config as code](https://docs.roark.ai/documentation/config-as-code/overview.md) yet. Create them in Studio or through the API.

***

## The Request Payload

With no body, Roark POSTs this payload as JSON. With a custom body, the same payload is what your [placeholders](#placeholders) read from.

```json theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
{
  "version": "1.0",
  "requestId": "3eb16131-4444-4fb1-8d84-6c4c6fce5043",
  "metric": { "id": "0b8e3e0e-1111-4c8e-9a51-3f1f3c9b2d10", "name": "PIN changed" },
  "call": {
    "id": "6f1d2c3b-5555-4a6e-8f7a-1b2c3d4e5f60",
    "projectId": "7a2e3d4c-6666-4b7f-9a8b-2c3d4e5f6a71",
    "callDirection": "INBOUND",
    "startedAt": "2026-10-26T14:00:02.000Z",
    "endedAt": "2026-10-26T14:03:06.000Z",
    "durationMs": 184000,
    "endedStatus": "CUSTOMER_ENDED_CALL",
    "externalId": "your-call-id",
    "agents": [
      {
        "id": "8b3f4e5d-7777-4c8a-ab9c-3d4e5f6a7b82",
        "endpoint": { "id": "9c4a5f6e-8888-4d9b-bcad-4e5f6a7b8c93", "environment": "production", "phoneNumberE164": "+14155550100" }
      }
    ],
    "customers": [{ "phoneNumberE164": "+14155550199", "label": null }],
    "properties": { "cardId": "card_123" }
  },
  "simulation": {
    "jobId": "ad5b6a7f-9999-4eac-8dbe-5f6a7b8c9da4",
    "runPlanJobId": "be6c7b8a-aaaa-4fbd-9ecf-6a7b8c9daeb5",
    "flow": { "id": "cf7d8c9b-bbbb-4ace-afd0-7b8c9daebfc6", "title": "Change my PIN" },
    "testCase": { "id": "d08e9dac-cccc-4bdf-b0e1-8c9daebfc0d7", "title": "Happy path", "isDefault": true },
    "variables": { "cardNumber": "4111111111111111", "newPin": "4821" },
    "persona": { "name": "Maria" }
  }
}
```

| Field | Description |
| :- | :- |
| `requestId` | Unique per check, also sent as the `X-Roark-Request-Id` header. Retries reuse it, so you can deduplicate on it |
| `metric` | The metric being checked |
| `call` | The call, with the same field names as the [Calls API](https://docs.roark.ai/api-reference/introduction.md). `externalId` is your own id if you sent one, and `properties` holds the call's [custom properties](https://docs.roark.ai/documentation/integrations/custom-integrations.md) |
| `simulation` | Set when the call is a Roark simulation, `null` on a live call. `variables` holds the run's [variables](https://docs.roark.ai/documentation/simulation-testing/variables.md) and `persona` the persona the simulated caller played |

Every request also carries `User-Agent: Roark-Metric-Check/1.0`, and `Content-Type: application/json` on a `POST`. Headers you set take precedence.

***

## Placeholders

`{{path}}` placeholders read from the payload above, for example `{{call.id}}`, `{{call.properties.cardId}}`, or `{{simulation.variables.newPin}}`. Roark checks every placeholder when you save, so a typo is caught before any call runs.

How a value goes in depends on where the placeholder sits:

| Where | How the value goes in | Example |
| :- | :- | :- |
| URL | URL-encoded | `https://api.example.com/cards/{{call.properties.cardId}}/pin` |
| Header | As text. A value with a line break is refused | `X-Card-Id: {{call.properties.cardId}}` |
| Body, inside quotes | As escaped text | `"cardId": "{{call.properties.cardId}}"` |
| Body, outside quotes | As JSON: objects stay objects, text is quoted for you | `"call": {{call}}` sends the whole call object |

If a call has no value for a placeholder, such as a run variable on a live call or a property the call doesn't carry, Roark doesn't send the request. The result is recorded as missing data, never as a fail, and the literal `{{...}}` is never sent.

> [!TIP]
>
> A check written for simulations, such as one that reads `{{simulation.variables.newPin}}`, records missing data on live calls instead of failing them. Use a [collector's conditions](https://docs.roark.ai/documentation/metrics/metric-collectors.md) to run it only where it applies.

***

## Answering the Check

Reply with HTTP `200` and this JSON:

```json theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
{ "passed": true, "reason": "PIN on card_123 was changed at 14:02:51 UTC." }
```

| Field | Type | Description |
| :- | :- | :- |
| `passed` | boolean | Whether the call did what it should |
| `reason` | string | Optional. Shown next to the result in Roark. Longer than 1000 characters is cut |

A minimal endpoint:

```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
app.post('/roark/checks/pin-changed', async (req, res) => {
  const { cardId, expectedPin } = req.body
  const card = await cards.get(cardId)

  if (card.pin === expectedPin) {
    return res.json({ passed: true, reason: `PIN on ${cardId} matches the new PIN.` })
  }
  res.json({ passed: false, reason: `PIN on ${cardId} was not changed.` })
})
```

### Errors and retries

Only a `200` with `passed` set is a verdict. Anything else is recorded as an **error** on the call, not a fail, so your pass rate reflects your agent and not your endpoint's uptime.

| Your endpoint | Result |
| :- | :- |
| `200` with `{ "passed": ... }` | Pass or Fail |
| Times out (30 seconds), can't be reached, or answers `5xx` | Retried twice with the same `requestId`, then an error |
| Any other status, including other `2xx` and `4xx` | Error, not retried |
| `200` with a body that isn't the contract | Error |

Roark shows your `reason`, never your raw response, on the call.

***

## Security

* **Authenticate with headers.** Add your own `Authorization` or API key header. Header values are redacted in Roark's request log.
* **Public URLs only.** The URL must be public `http(s)`. Roark checks it when you save, again after placeholders are filled in, and once more against the resolved address when it connects, so a public hostname that points to a private address is refused.
* **No redirects.** Roark doesn't follow redirects, so point the URL at the final endpoint.
* **Bounded responses.** Responses over 64 KB are not read.

***

## Limits

* API checks run on calls only, not chats.
* An API check has no variants: the request is the metric, so edit it on the metric itself.

***

## What's Next

> **[Studio](https://docs.roark.ai/documentation/metrics/studio.md)**
>
> Author and test an API check before it runs on calls

> **[Collectors](https://docs.roark.ai/documentation/metrics/metric-collectors.md)**
>
> Run your API check automatically on matching live calls

> **[Run Plans](https://docs.roark.ai/documentation/simulation-testing/run-plans.md)**
>
> Attach the check to a simulation and gate a run on it

> **[Variables](https://docs.roark.ai/documentation/simulation-testing/variables.md)**
>
> Pass run variables your check can compare against


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.