> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Live Bench

> Roark’s standard voice-agent benchmark: task completion, turn-taking, latency and robustness in realistic conditions

## Why this is a common failure point

Every team tests its agent differently, so "our agent is good" means nothing outside the team. Comparing two models, two voice providers or two versions of your own agent needs the same calls, the same conditions and the same grading every time. Otherwise a better score may just be an easier test.

## How Roark tests it

Live Bench is a fixed battery of calls with standardised flows, conditions and metrics. Nothing about it can be configured, so a score means the same thing across agents and against Roark's published baselines. You pick the agent to put on the line.

***

## Setting it up

The suite runs four batteries:

| Battery                | What it covers                                                                              |
| :--------------------- | :------------------------------------------------------------------------------------------ |
| **Duplex dynamics**    | Barge-in, backchannels, mid-turn hesitation, and corrections spoken over a read-back        |
| **Task pursuit**       | A three-part request with a change of mind, plus error-prone names, numbers and codes       |
| **Robustness**         | Accented and broken English over street, car and cafe noise; a caller who starts frustrated |
| **Safety spot-checks** | The two highest-signal adversarial probes: prompt injection and PII extraction              |

***

## What it measures

The template seeds these, and you can add or remove metrics in **Advanced** before running.

| Measure                                 | What it tells you                                                                                                                                      |
| :-------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Instruction Follow (check)**          | How well the agent followed its instructions, 1 to 5. The check passes at 3 or above.                                                                  |
| **Flow Adherence (check)**              | How closely the conversation followed the flow, 1 to 5. The check passes at 4 or above.                                                                |
| **User Effort Score (check)**           | How hard the caller had to work to get what they wanted, 1 to 5. The check passes at 4 or below.                                                       |
| **Loop Count (check)**                  | Conversational loops the agent got stuck in. The check allows none.                                                                                    |
| **Agent Cutoff Count (check)**          | Times the agent cut the caller off. The check allows one per call.                                                                                     |
| **Prompt Injection Resistance (check)** | Whether the agent resisted attempts to override its instructions.                                                                                      |
| **PII Handling (check)**                | How properly the agent handled personal data. Passes at 3 or above.                                                                                    |
| **Latency breakdown**                   | Time to first word, response time, latency spikes, and each stage of the turn: transcription, endpointing, the model, tool calls and speech synthesis. |
| **Turn-taking**                         | Interruption appropriateness, barge-in handling, overtalk and failed barge-ins.                                                                        |

***

## What to look for

* **The latency bottleneck.** It names the stage that owned most of each slow turn, so you know whether to change the transcriber, the model or the voice.
* **Compare like with like.** Live Bench is built for A/B comparisons: run it on both versions of the agent and read the differences, not the absolute scores.

***

## Over the API

Live Bench is not available over the API yet.