Skip to main content

​
Why this is a common failure point

Callers don’t speak at one speed. A slow speaker pauses mid-sentence, and an agent with a short end-of-turn timeout jumps in and cuts them off. A fast speaker runs words together and the transcriber drops or merges them. Both produce an agent that answers something the caller didn’t finish saying.

​
How Roark tests it

Speech pace handling runs the same flow with callers at each pace you pick, from very slow to very fast, and tells you whether any pace makes your agent misunderstand the caller or fail the call, and whether the agent adapts its own pace. It is a property sweep: every value runs the same flow with everything else held the same, and the report tells you which values did worse than the rest and which differences are just chance.

​
The values

Values are ordered from slowest to fastest, which is also how the report lists them.

​
What it measures

The template seeds these, and your flow’s own metrics run alongside them. Only checks can mark a value Worse on; the other measures explain why.

​
What to look for

  • Slow callers and turn-taking. Very slow speech has long pauses, and an agent that answers into a pause cuts the caller off.
  • Fast callers and recognition. Very fast speech runs words together, so failures there usually come from transcription.
  • Does the agent adapt? Speech Pace Delta shows whether it slows down for a slow caller or keeps its own pace.

​
Setting one up

Pick the flow to test, then the values to run it across. Every value starts selected; deselect the ones you don’t need. Each value runs your flow again, so the values you keep are the cost of the run. Compared against sets the baseline: a reference value the report pins first, such as silence for background noise. The template suggests the natural one, and you can change it or choose none. It doesn’t change the verdict: whether a value did worse is always decided against every other value combined. Iterations live in the sweep panel rather than in Advanced. Each value runs every attached test case once per iteration:
Calls per value = test cases x iterations
Each value needs at least 5 calls to be compared with the others, and the Iterations field won’t go below the count that reaches it. It starts at 6 calls per value, so one call excluded after the run doesn’t drop a value out of the comparison. More calls let the report see smaller differences. For a sweep of 10 values:
Below 5 calls a value can never be flagged, however badly it does, which is why the minimum is enforced rather than suggested. Over the API, a sweep template defaults iterationCount to 6 and refuses a plan with fewer than 5 calls per value with a 400.

​
How the results are decided

The sweep’s table leads the run report. Its Compared with the rest column gives each value one result: How “too large to be chance” is decided:
  • Each value is compared with all the other values combined, one check at a time, never with a single other value. Two values that happen to land at opposite ends are not a finding on their own.
  • A sweep makes many comparisons at once (every value on every check), and some will look different by luck alone. The report accounts for that: of all the values it flags in a run, at most 1 in 10 is expected to be a fluke.
  • A value is flagged only when it does worse. Doing better than the rest is not a problem to fix.
Checks passed shows each value’s pass rate with the number of calls behind it, so 100% of two calls never reads like 100% of twenty. A lower rate with No difference beside it means the gap is not larger than chance. Fails, not speech pace specific. A check that fails on at least 20% of calls with no value standing out is listed under this heading. It is a real problem, just not one speech pace caused, so changing speech pace won’t fix it. Select the check to open its evidence. The executive summary follows the same rules: it only blames the property for failures the table marks Worse on. Hover How this works under the table for this run’s own numbers.