Thresholds

​

Overview

Thresholds turn raw metric values into clear pass/fail outcomes. When you add a threshold to a metric, Roark mints a derived boolean metric that automatically evaluates whether each conversation meets your criteria. For example, a threshold of >= 7 on Customer Satisfaction produces a Pass whenever the score is 7 or above, and a Fail otherwise.

---

​

How Thresholds Work

Each threshold is a derived metric in its own right:

1. The source metric is collected as usual (e.g., a satisfaction score of 8)
2. The threshold compares the value against your condition (e.g., >= 7)
3. A boolean result is produced with the labels Pass and Fail
4. The derived metric is stored alongside the original, so both show up in reporting

The derived metric is named after its rule — Customer Satisfaction >= 7 — and uses the THRESHOLD calculation type with a Boolean output. Because it’s a real metric definition, the same threshold can be reused across collectors.

---

​

Adding a Threshold

Thresholds can be configured from several places in Roark:

• Collectors — Each picked metric in the collector editor has a Pass/Fail Threshold(s) block with an Add pass/fail threshold button
• Studio — In Evaluate mode, configure thresholds inline per metric before running a battery of evals
• Simulation Run Plans — Set pass/fail checks for simulation testing

The configuration is the same everywhere: each threshold row reads “Pass if [operator] [value]”.

​

Reusing Saved Thresholds

Because thresholds are metric definitions, ones you’ve already created appear as read-only rows with an Include if checkbox — tick it to opt an existing threshold into the current collector. If you try to add a rule that matches a saved threshold, Roark warns “Matches an existing threshold — it won’t be created again” and reuses the existing one instead of creating a duplicate.

---

​

Operators

The available operators depend on the metric’s output type.

​

Numeric Types (Scale, Numeric, Count)

​

Categorical Types (Boolean, Classification)

---

​

API-Only Options

The UI keeps threshold rows simple — an operator and a value. The REST API supports two additional options when creating thresholds programmatically.

​

Aggregation Modes

When a metric produces multiple values per call (e.g., a per-segment metric that fires on every turn), the aggregationMode field determines how those values are combined before the comparison is applied.

​

Participant Role Filtering

For metrics with PER_PARTICIPANT scope, the sourceParticipantRole field narrows the threshold to a specific speaker: AGENT, CUSTOMER, SIMULATED_CUSTOMER, or BACKGROUND_SPEAKER. This is useful when a metric like sentiment is tracked for both speakers but you only want to set a threshold on the agent’s performance.

---

​

Examples

---

​

What’s Next