> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# How the agent handles a bad line

> Run one flow on a clear line and on each bad one, and see what every step down costs the agent

## Why this is a common failure point

A phone line that drops audio does not sound like noise. It cuts syllables, swallows a digit in an account number, or loses the word "not". The agent either asks the caller to repeat themselves, guesses, or answers something nobody said. Lines also fail one way: an agent that hears the caller perfectly can still be talking to someone who missed half its question.

## How Roark tests it

**How the agent handles a bad line** runs the same flow with the caller on each [phone line](https://docs.roark.ai/documentation/simulation-testing/network-conditions.md) you pick, from clear to one-way loss. The caller, their sound and the conversation are identical in every value: only the line changes, so the report shows what each one costs your agent.

It is a **property sweep**: every value runs the same flow with everything else held the same, and the report tells you which values did worse than the rest and which differences are just chance.

> [!NOTE]
>
> Only an agent reached on a phone endpoint has a line to degrade. Run this template against a phone endpoint; on any other channel every value sounds the same.

***

## The values

| Value | What it loses |
| :- | :- |
| Clear | Baseline: no loss. Always runs. |
| Minor loss | 1% at random, both ways |
| Cell edge | 5% in 80 ms gaps, both ways |
| Poor | 10% in 120 ms gaps, both ways |
| One way | 8% in 80 ms gaps, toward the agent only |

Custom lines are not a sweep value. To test one, put it on an [environment](https://docs.roark.ai/documentation/simulation-testing/environments.md) and attach a variant that uses it.

***

## What it measures

The template seeds these, and your flow's own metrics run alongside them. Only checks can mark a value **Worse on**; the other measures explain why.

| Measure | What it tells you |
| :- | :- |
| **Comprehension Failure (check)** | The agent misunderstood or misheard the caller. The check fails any call where it did. |
| **Redundant Question Count (check)** | The agent asked for something the caller already gave. The check allows one per call. |
| **Missed Response Count** | Moments the agent should have answered and did not, often because the audio that needed an answer never arrived. |
| **Call Outcome** | Whether the call ended where it should. |

To see what the carrier measured on each call, add the [Telephony metrics](https://docs.roark.ai/documentation/simulation-testing/network-conditions.md#what-the-carrier-measured) (Network MOS, Packet Loss and Max Jitter) in **Advanced**.

***

## What to look for

* **Gaps against random loss.** Minor loss drops single packets; Cell edge and Poor drop them in bursts. An agent that copes with the first and breaks on the second is losing whole words, not crackle.
* **The one-way line.** One way degrades only what the agent hears. Compare it with Cell edge, which degrades both directions at a similar rate, to see whether failures come from the agent mishearing or the caller mishearing.
* **Repeated questions.** A rise in Redundant Question Count on bad lines is the agent asking again for what it lost. Some of that is good behaviour; a check that fails on every bad line is not.
* **Noise and lines together.** A bad line and background noise fail in different ways. Run [Background noise robustness](https://docs.roark.ai/documentation/simulation-testing/templates/background-noise-robustness.md) as well, or try the ready-made bad-line environments, which pair a line with a matching sound.

***

## Setting one up

Pick the flow to test, then the lines to run it on. Every line starts selected; deselect the ones you don't need. **Clear** always runs and is the baseline every other line is read against, so it can't be deselected. Each line runs your flow again, so the lines you keep are the cost of the run.

Each value needs **at least 5 calls** to be compared with the others. Iterations live in the sweep panel, and each line runs every attached test case once per iteration:

> **Calls per line = test cases x iterations**

The [property sweep setup](https://docs.roark.ai/documentation/simulation-testing/templates/background-noise-robustness.md#setting-one-up) explains how calls per value relate to the size of difference the report can detect.

***

## How the results are decided

The sweep's table leads the run report. Its **Compared with the rest** column gives each value one result:

| Result | What it means |
| :- | :- |
| **Worse on** *check* | The value failed that check more often than all the other values combined, by a gap too large to be chance. |
| **No difference** | Any gap is within what chance produces with this many calls. That is not proof of no effect: a small run can only see large gaps, and the report says how large when that matters. |
| **Too few calls** (*n*) | Fewer than 5 counted calls, so the value is shown but not compared. |
| **Not graded** | The value ran, but none of its calls were graded. |

How "too large to be chance" is decided:

* Each value is compared with **all the other values combined**, one check at a time, never with a single other value. Two values that happen to land at opposite ends are not a finding on their own.
* A sweep makes many comparisons at once (every value on every check), and some will look different by luck alone. The report accounts for that: of all the values it flags in a run, **at most 1 in 10 is expected to be a fluke**.
* A value is flagged only when it does **worse**. Doing better than the rest is not a problem to fix.

**Checks passed** shows each value's pass rate with the number of calls behind it, so 100% of two calls never reads like 100% of twenty. A lower rate with **No difference** beside it means the gap is not larger than chance.

**Fails, not line quality specific.** A check that fails on **at least 20% of calls** with no value standing out is listed under this heading. It is a real problem, just not one line quality caused, so changing line quality won't fix it. Select the check to open its evidence.

The executive summary follows the same rules: it only blames the property for failures the table marks **Worse on**. Hover **How this works** under the table for this run's own numbers.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.