Conversation quality

​

Why this is a common failure point

An agent can complete every task and still be unpleasant to talk to. It asks for the account number twice, leaves a silence long enough for the caller to say “hello?”, talks over them, or answers a frustrated caller in the same bright tone it uses for everyone. These don’t fail a task, so task-based tests pass while callers hang up annoyed. Quality problems also hide in averages: one bad turn in an otherwise fine call is what the caller remembers.

​

How Roark tests it

Conversation quality runs your flows with realistic, unscripted callers and grades the experience rather than the outcome: how much effort the caller had to spend, whether the agent repeated itself or missed a turn, how it handled the rhythm of the conversation, and how the caller felt by the end.

---

​

Setting it up

Attach any flows through the inline Flows panel. The template works best on improv flows, where the caller behaves like a real person rather than following a script.

---

​

What it measures

The template seeds these, and you can add or remove metrics in Advanced before running.

---

​

What to look for

• Effort before sentiment. Callers forgive a flat tone; they don’t forgive repeating themselves. User effort and redundant questions are the first things to fix.
• Response time spikes. A long pause mid-call reads as a dropped line. Look at the slowest turns, not the average.
• Read the calls behind a low score. Every metric links to the turns that produced it.

---

​

Over the API

Run it with POST /v1/simulation/run and "template": "conversation-quality". GET /v1/simulation/template lists what each template accepts.