Skip to main content
Metric definitions describe what to measure and how. Roark comes with built-in system metrics that work out of the box, and you can create your own custom metrics tailored to your business needs.
The Metrics page: a Collectors strip across the top and the metric Library grouped by package, with All / System / Custom filters

Choose an Engine

Custom metrics let you measure anything specific to your use case: task completion, compliance checks, quality scoring, or business KPIs. Every custom metric runs on one of four engines. Pick the engine by where the answer lives:

LLM Judge

Describe what to measure in plain language

Patterns

Detect a trigger and what follows it

Formulas

Compute a metric from other metrics

API Checks

Ask your own endpoint whether the call passed
Patterns, formulas, and API checks don’t use Roark’s evaluation models, so they aren’t billed like LLM Judge metrics.

Configuration Steps

1

Define the Metric in Studio

Open Studio in Author mode. Name your metric and pick an engine (locked after the first save). Then fill in what that engine needs: the prompt and output type for an LLM Judge, the triggers and outcome for a Pattern, the expression for a Formula, or the request for an API check. LLM Judge metrics and API checks can also be defined in code.
Studio Author mode: the numbered Engine, Output (with range labels), Instructions, and Group sections, plus a Tests rail of scored test calls
2

Test Against Real Calls

Use the test rail in Studio to run your metric against representative calls and validate it produces the results you expect. Iterate until you’re satisfied.
3

Add to a Collector or Run Plan

Attach the metric to a collector for automated collection on incoming calls, or to a simulation run plan for testing.

SDK Reference

All API endpoints require authentication. Generate an API key to get started.

List Metric Definitions

Retrieve the first page of metric definitions available for your project. Each page contains up to 500 definitions. If pagination.hasMore is true, use pagination.nextCursor as the after query parameter on GET /v1/metric/definitions to retrieve the next page. Results are ordered by immutable definition ID. Treat cursors as opaque.

Get Call Metrics

Retrieve all metrics for a specific call:
The response groups metrics by definition, with each metric containing an array of values:

Understanding Metric Values

Confidence Scores:
  • All metrics include a confidence field (0-1)
  • Deterministic metrics (like word count, duration) have confidence = 1.0
  • AI-powered metrics include the model’s confidence level
Value Reasoning:
  • For AI-computed metrics, the valueReasoning field provides explanation
  • Useful for understanding why a metric was scored a certain way
  • Example: “The agent verified identity using two-factor authentication as mentioned in segment 3”
Segment Context:
  • When context is SEGMENT, the segment field contains the specific utterance
  • When context is SEGMENT_RANGE, both fromSegment and toSegment are included
  • All segment objects include the full text and timing information

Best Practices

Use the built-in system metrics first. They cover performance, sentiment, interruptions, compliance, and more with no setup. Add custom metrics for business-specific needs.
Always validate custom metrics in Studio on representative calls before attaching them to collectors.
Instead of creating one complex LLM prompt that tries to measure everything, break it into focused metrics and combine them with a formula.
Track both agent and customer metrics for complete conversation understanding.

What’s Next

System Metrics Reference

Browse all 65+ built-in metrics powered by specialized models

Studio

Author and test metrics interactively before deploying

Thresholds

Define pass/fail criteria for your metrics

Collectors

Automate metric collection with conditions-based rules