> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Conversation quality

> Measure how natural, empathetic and helpful your agent sounds across realistic, unscripted conversations

## Why this is a common failure point

An agent can complete every task and still be unpleasant to talk to. It asks for the account number twice, leaves a silence long enough for the caller to say "hello?", talks over them, or answers a frustrated caller in the same bright tone it uses for everyone. These don't fail a task, so task-based tests pass while callers hang up annoyed.

Quality problems also hide in averages: one bad turn in an otherwise fine call is what the caller remembers.

## How Roark tests it

Conversation quality runs your flows with realistic, unscripted callers and grades the experience rather than the outcome: how much effort the caller had to spend, whether the agent repeated itself or missed a turn, how it handled the rhythm of the conversation, and how the caller felt by the end.

***

## Setting it up

Attach any flows through the inline Flows panel. The template works best on improv flows, where the caller behaves like a real person rather than following a script.

***

## What it measures

The template seeds these, and you can add or remove metrics in **Advanced** before running.

| Measure                              | What it tells you                                                                                |
| :----------------------------------- | :----------------------------------------------------------------------------------------------- |
| **User Effort Score (check)**        | How hard the caller had to work to get what they wanted, 1 to 5. The check passes at 4 or below. |
| **Instruction Follow (check)**       | How well the agent followed its instructions, 1 to 5. The check passes at 3 or above.            |
| **Redundant Question Count (check)** | The agent asked for something the caller already gave. The check allows one per call.            |
| **Frustration Score**                | How frustrated the caller became, 1 to 5.                                                        |
| **Sentiment Score**                  | The caller's sentiment, 1 (negative) to 9 (positive).                                            |
| **Comprehension Failure**            | Whether the agent misunderstood or misheard the caller.                                          |
| **Missed Response Count**            | Moments the agent should have answered and did not, so the caller had to repeat themselves.      |
| **Interruption Appropriateness**     | Whether each time the agent cut in was the right moment.                                         |
| **Talk-to-Listen Ratio**             | How much of the call each side spent talking.                                                    |
| **Overtalk Ratio**                   | How much of the call both sides spoke at once.                                                   |
| **Response Time**                    | Time between speaking turns.                                                                     |
| **Call Outcome**                     | Whether the call ended where it should.                                                          |

***

## What to look for

* **Effort before sentiment.** Callers forgive a flat tone; they don't forgive repeating themselves. User effort and redundant questions are the first things to fix.
* **Response time spikes.** A long pause mid-call reads as a dropped line. Look at the slowest turns, not the average.
* **Read the calls behind a low score.** Every metric links to the turns that produced it.

***

## Over the API

Run it with `POST /v1/simulation/run` and `"template": "conversation-quality"`. `GET /v1/simulation/template` lists what each template accepts.