Skip to main content

Overview

A collector runs a set of metrics on the calls (or chats) it matches. Instead of triggering metric collection by hand, you define the segment of conversations you care about (by agent, source, or property) and Roark scores every matching conversation as it comes in. Define a collector in code (recommended), or build it in the dashboard.
See the config as code reference for every field, including per-collector metric settings. Collectors live under Measure → Metrics: the Metrics page shows a Collectors strip at the top, and Manage collectors opens the full list at /metrics/collectors.
The Collectors list at /metrics/collectors: a System collector (Call Monitoring) with a shield badge and a static Active status, above custom collectors showing their segment summary, Runs metric chips, an Edit chip, and an Active or Paused toggle
A collector is made of four things:
  1. Name: how it appears in the collectors list
  2. Modality: Call or Chat. A collector fires for exactly one modality, and this is locked after creation
  3. Conditions (optional): the segment of conversations it matches. Leave empty to match all calls (or all chats)
  4. Metrics: the metrics to run on every match, each with optional pass/fail thresholds

Creating a Collector

Build it in the dashboard

1

Start a new collector

On /metrics/collectors, click New collector. The editor reads as a sentence: “When a call matches…” Click the chip to switch between Call and Chat. Once the collector is created, the modality is locked.
2

Define the segment

Add conditions to target specific conversations, or leave the conditions empty to match everything. Use Add condition and Add OR condition to build a group, and Add AND condition group to require a second group to match as well.
3

Pick the metrics

Under Evaluate these metrics, add the metrics to run. Each picked metric runs on every conversation that matches the scope above. Where a metric has multiple variants, choose which variant’s prompt and config apply. Some metrics take a setting of their own here, such as Properties to verify for Property Verification (see per-metric settings).
4

Add thresholds (optional)

On any picked metric, click Add pass/fail threshold to turn its raw value into a pass/fail outcome, for example, “Pass if ≥ 7”.
5

Create and name it

Click Create collector. Roark suggests a name built from the metrics and conditions. Keep it or edit it, then confirm. Incoming conversations that match your conditions are evaluated from then on.
The collector editor: a 'When a call matches…' conditions section with one group matching Agent is Billing Assistant OR Source is Vapi, above an 'Evaluate these metrics' section with the Disclosure Completeness scale metric showing a Pass if ≥ 7 pass/fail threshold and an Add pass/fail threshold button

Conditions

Conditions scope a collector to a segment of conversations. Groups combine with AND; conditions within a group combine with OR. An empty condition set matches every call (or chat) in the project. Property conditions support these operators:
A group like “Agent is Billing Bot OR Agent is Sales Bot” AND “Source is Vapi” runs the collector’s metrics only on Vapi calls handled by either agent.

Pass/Fail Thresholds

Thresholds turn a metric’s raw value into a pass/fail check, configured inline on each picked metric:
  • Add pass/fail threshold adds a row that reads “Pass if [operator] [value]” (for example, Pass if ≥ 0.7).
  • Thresholds that already exist for the metric appear as read-only rows with an Include if checkbox, so you can opt an existing threshold into this collector instead of creating a duplicate. If a new row matches an existing threshold, Roark warns you and dedupes it. It won’t be created again.
Each saved threshold is a derived metric in its own right (named like “Empathy ≥ 7”, scored Pass/Fail), added to the collector alongside the source metric. Thresholds are available for Scale, Numeric, Count, Boolean, and Classification outputs.

Thresholds Guide

Learn about operators and how derived threshold metrics work

Active and Paused

Every custom collector has a status toggle in the list: Active collectors evaluate matching conversations; Paused collectors don’t. Pause a collector instead of deleting it if you might need it again. Deleting a collector stops it from collecting its metrics, but the metric definitions themselves are unaffected.

System Collectors

Roark ships system collectors, the baseline coverage that keeps core metrics running on all your calls and chats. They’re marked with a System shield badge and sort first in the list.

Per-Agent Collectors

Each agent’s detail page has a Collectors section listing the collectors that target that agent. These are the same collectors (just scoped by an Agent condition) so a collector created from the agent page shows up in the main /metrics/collectors list too, and vice versa.

API and SDK Access

Collectors are fully manageable programmatically via the Node.js SDK and REST API.
The API keeps the older metric policy naming: the SDK client is client.metricPolicy.* and the REST endpoints are /v1/metric/policies. A “policy” in the API is exactly a collector in the dashboard.

Create it with the SDK

Per-metric settings

Some metrics take a setting that applies only in this collector, the same setting the dashboard shows under the metric when you pick it (for example Properties to verify). Pass it as settings on the metric:
  • A metric takes at most one setting, and only the one listed for it; anything else is a 400 naming the setting the metric takes.
  • Responses return each metric’s settings, or null when it runs with its defaults.
  • An update that sends metrics replaces the collector’s whole metric list, and a metric sent without settings goes back to its defaults. Send each metric’s settings again to keep them; a fetched metrics list can be sent back as is.
  • In config as code the same settings go on the metric entry in YAML.

Condition fields

GREATER_THAN_OR_EQUALS and LESS_THAN_OR_EQUALS correspond to the dashboard’s “is at least” and “is at most” operators. Example: collector for Vapi calls only

List, get, update, delete

System collectors (type: "SYSTEM" in API responses) cannot be modified or deleted.

Best Practices

Create a collector with no conditions to run key metrics on all calls, then add targeted collectors with conditions for specific segments.
If you only need certain metrics for specific agents or sources, scope the collector so you’re not scoring conversations you don’t care about.
Use names that say what the collector targets and measures. The auto-suggested name (metric names plus a condition summary) is a good starting point.
Pause a collector if you might need it again later. Deleting stops collection but leaves the metric definitions intact.

What’s Next

Custom Metrics

Learn about metric types, scopes, and how to create custom metrics

Studio

Author and test metrics, then bind them to a collector from the editor

Thresholds

Turn raw metric values into pass/fail outcomes

Metric Collection Jobs

Run metrics on demand for specific calls