Patterns

A pattern metric watches a call line by line. When a trigger happens, it looks at the lines just after it for an outcome, and records whether the pattern happened or how many times. “Did the agent talk over a caller who was frustrated?” is a pattern: the trigger is the caller’s frustration, the outcome is an agent interruption within the next few lines. So is “did the agent recover after a misunderstanding” or “how often did a slow response follow a question”. Patterns are built from metrics Roark already records on each line, so they don’t call a model: the same call always gives the same result, and patterns aren’t billed like LLM Judge metrics.

Quickstart

Patterns aren’t a config as code kind yet. Create the pattern once with the SDK under a stable slug, then collect it from config by that slug. You can also build it in Studio.

Create it with the SDK

Create the metric with calculationType: "PATTERN". Sources are referenced by metric definition id, so look them up first:

import Roark from '@roarkanalytics/sdk'

const client = new Roark({ bearerToken: process.env.ROARK_API_BEARER_TOKEN })

const { data: definitions } = await client.metric.listDefinitions()
const idOf = (slug: string) => definitions.find((d) => d.metricId === slug)!.id

const metric = await client.metric.createDefinition({
  calculationType: 'PATTERN',
  name: 'Talked over a frustrated caller',
  slug: 'talked_over_frustrated_caller',
  operation: 'PATTERN_EXISTS',
  windowMode: 'segments',
  trigger: {
    sourceMetricDefinitionId: idOf('frustration_score'),
    operator: 'GREATER_THAN_OR_EQUALS',
    thresholdValue: '4',
    sourceParticipantRole: 'CUSTOMER',
  },
  outcome: {
    sourceMetricDefinitionId: idOf('interruption'),
    operator: 'EQUALS',
    thresholdValue: 'true',
    sourceParticipantRole: 'AGENT',
    windowAfter: 3,
  },
})

import os
from roark_analytics import Roark

client = Roark(bearer_token=os.environ.get("ROARK_API_BEARER_TOKEN"))

definitions = client.metric.list_definitions().data
id_of = lambda slug: next(d.id for d in definitions if d.metric_id == slug)

metric = client.metric.create_definition(
    calculation_type="PATTERN",
    name="Talked over a frustrated caller",
    slug="talked_over_frustrated_caller",
    operation="PATTERN_EXISTS",
    window_mode="segments",
    trigger={
        "source_metric_definition_id": id_of("frustration_score"),
        "operator": "GREATER_THAN_OR_EQUALS",
        "threshold_value": "4",
        "source_participant_role": "CUSTOMER",
    },
    outcome={
        "source_metric_definition_id": id_of("interruption"),
        "operator": "EQUALS",
        "threshold_value": "true",
        "source_participant_role": "AGENT",
        "window_after": 3,
    },
)

curl -X POST https://api.roark.ai/v1/metric/definitions \
  -H "Authorization: Bearer $ROARK_API_BEARER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "calculationType": "PATTERN",
    "name": "Talked over a frustrated caller",
    "slug": "talked_over_frustrated_caller",
    "operation": "PATTERN_EXISTS",
    "windowMode": "segments",
    "trigger": {
      "sourceMetricDefinitionId": "<frustration_score id>",
      "operator": "GREATER_THAN_OR_EQUALS",
      "thresholdValue": "4",
      "sourceParticipantRole": "CUSTOMER"
    },
    "outcome": {
      "sourceMetricDefinitionId": "<interruption id>",
      "operator": "EQUALS",
      "thresholdValue": "true",
      "sourceParticipantRole": "AGENT",
      "windowAfter": 3
    }
  }'

The response is the metric definition, with your configuration echoed back under pattern.

Field	Type	Required	Description
`name`	string	Yes	Name of the metric (1-100 characters)
`operation`	string	Yes	`PATTERN_EXISTS` (Boolean), `PATTERN_COUNT` (Count), or `OUTCOME_AGGREGATE` (Numeric, see below)
`trigger`	object	One of	A single trigger condition
`triggers`	object[]	One of	Several trigger conditions, combined with `triggerCombinator`
`triggerCombinator`	string	Conditional	`AND` or `OR`. Required with more than one trigger
`outcome`	object	Yes	The outcome condition, with its window
`windowMode`	string	No	`seconds` (default) or `segments`
`slug`	string	No	Stable identifier. Generated from the name if omitted
`analysisPackageId`	string	No	Package to add the metric to. Defaults to your project’s Custom Metrics package

Each trigger and the outcome take:

Field	Type	Required	Description
`sourceMetricDefinitionId`	string	Yes	The source metric’s definition id
`operator`	string	Yes	`EQUALS`, `NOT_EQUALS`, `GREATER_THAN`, `GREATER_THAN_OR_EQUALS`, `LESS_THAN`, `LESS_THAN_OR_EQUALS`
`thresholdValue`	string	Yes	The value to compare with: a number, or `true` / `false` for a Boolean source
`sourceParticipantRole`	string	No	`AGENT` or `CUSTOMER`, for a per-participant source
`sourceVariantId`	string	No	A variant of the source metric to read
`windowAfter`	integer	Outcome only	How far after the trigger to look, in `windowMode` units
`windowBefore`	integer	No, outcome only	How far before the trigger to look. Default `0`

A request that references the same source metric twice, across the triggers and the outcome, is refused with 400. PUT /v1/metric/definitions/{idOrSlug} changes a pattern’s name. Its triggers, outcome, and window are fixed once it’s created: to change them, create a new pattern.

Collect it from config

Once the pattern exists, list its slug in a collector next to your other metrics, and Roark scores it on every matching call. Roark collects the pattern’s source metrics along with it, so you don’t list them:

import { defineCollector, defineConfig } from '@roarkanalytics/sdk/lib/config';

const frontdeskQuality = defineCollector({
  name: 'frontdesk-quality',
  modality: 'call',
  status: 'ACTIVE',
  metrics: ['talked_over_frustrated_caller'],
});

export default defineConfig({
  resources: [frontdeskQuality],
});

# yaml-language-server: $schema=https://roark.ai/roark-config.schema.json
kind: collector
name: frontdesk-quality
modality: call
status: ACTIVE
metrics:
  - talked_over_frustrated_caller

roark config apply roark.config.ts

Apply fails with unknown metric "talked_over_frustrated_caller" until the pattern has been created.

Build it in Studio

Open Studio in Author mode and pick the Pattern engine. The engine is locked after the first save.

Studio Author mode with the Pattern engine selected: two triggers joined by AND, Frustration Score >= 4 and Vocal Activation >= 0.7, both for the Customer, then within 3 segments, check for Interruption equal to True for the Agent, with Did it happen? selected as the result

If this happens

Pick a trigger metric, an operator, and a value. For a per-participant metric, choose whose value counts: Agent or Customer. Click Add condition for more triggers, and choose AND or OR between them.

Then within

Set the window, in segments (1 to 20, default 3) or seconds (1 to 300, default 30).

Check for

Pick the outcome metric and the value it should reach. Each trigger and the outcome must use a different source metric.

What does this metric return?

Did it happen? gives a Boolean per call. How many times? gives a Count per call.

How It Works

Roark walks the call's segments in order

A segment is one line of the transcript. Overlapping speech and system actions, such as a keypress or a hang-up, are segments too.

Each segment is checked against the triggers

A trigger compares a source metric’s value on that segment with a threshold, for example Frustration Score >= 4 for the customer. With several triggers, AND needs all of them to hold on the same segment, OR needs any one.

The window after a trigger is searched for the outcome

When the triggers hold, Roark looks at the segments inside the window, for example the next 3 segments, for one where the outcome holds. The trigger’s own segment isn’t part of the window.

The call gets one value

Did it happen? is true if any trigger found its outcome. How many times? counts the trigger segments that did. The segments that matched are marked on the call’s timeline as the metric’s evidence.

Choosing Source Metrics

A pattern reads the values a metric records on segments or turns. Metrics that only record one value per call never match a trigger or an outcome. Good sources from the system metrics:

Metric	Slug	Recorded per	Values
Frustration Score	`frustration_score`	Segment	1 to 5
Sentiment Score	`sentiment_score`	Segment	1 to 9
Vocal Activation	`vocal_activation`	Segment	0 to 1
Interruption	`interruption`	Segment	`true` / `false`
Comprehension Failure	`comprehension_failure`	Turn	`true` / `false`
Response Time	`response_time`	Turn	Milliseconds

Your own LLM Judge metrics work too, when they record values per segment or turn. Conditions compare numbers. Numeric, Count and Scale values are compared as they are, and a Boolean is true (1) or false (0). Classification and Text metrics can’t be used as a source: their values never match.

Windows

Mode	The window covers
`segments`	The next `windowAfter` segments after the trigger, and the `windowBefore` segments before it
`seconds`	Every segment that starts up to `windowAfter` seconds after the trigger segment starts, or `windowBefore` seconds before it

The trigger’s own segment is never part of its window, so a pattern always relates two different lines.

Averaging the outcome

OUTCOME_AGGREGATE (API only) ignores the outcome’s operator and threshold. It averages the outcome metric’s values across every trigger’s window and records the result as a Numeric. Use it for questions like “how fast did the agent respond after a frustrated line”: trigger on Frustration Score >= 4, aggregate Response Time.

When Data Is Missing

If no trigger metric recorded any values on the call, or the outcome metric recorded none, the pattern records N/A with the reason, rather than false.
A call with no transcript segments gets no value.
Make sure the source metrics run on the same calls: add them to the same collector or run plan as the pattern.

What’s Next

System Metrics

Find per-segment metrics to build patterns from

Formulas

Combine call-level metrics into one score

Studio

Build a pattern and test it on real calls

Collectors

Run the pattern automatically on live calls

Getting Started

Simulations

Observability

Metrics

Autoimprove

Recipes

Integrations

Enterprise

Resources

Quickstart

Create it with the SDK

Collect it from config

Build it in Studio

How It Works

Choosing Source Metrics

Windows

Averaging the outcome

When Data Is Missing

What’s Next

System Metrics

Formulas

Studio

Collectors

​Quickstart

​Create it with the SDK

​Collect it from config

​Build it in Studio

​How It Works

​Choosing Source Metrics

​Windows

​Averaging the outcome

​When Data Is Missing

​What’s Next

System Metrics

Formulas

Studio

Collectors

Quickstart

Create it with the SDK

Collect it from config

Build it in Studio

How It Works

Choosing Source Metrics

Windows

Averaging the outcome

When Data Is Missing

What’s Next