All system metrics listed below are powered by specialized models purpose-built for voice AI analysis. This means they are faster, more consistent, and more cost-effective than general-purpose LLM evaluation.
Metric Types Overview
System metrics are computed by purpose-built models that analyze audio and transcript signals: performance, interruptions, sentiment, compliance, call quality. When you need something they don’t cover, create a custom metric on one of four engines:
System metrics make good inputs: per-segment metrics such as Frustration Score and Interruption feed patterns, and call-level ones such as Task Completion feed formulas.
System Metrics Reference
All system metrics below are collected automatically when included in an analysis package. Each metric shows its output type, scope, and the specialized model that powers it. Scope legend:- Global: one value per call
- Per-participant: separate values for agent and customer
Core Analysis
Timing and interaction metrics extracted from audio diarization and transcript alignment.Powered by Roark Vibe, our core voice analysis model.
Sentiment & Emotion
Emotion and sentiment analysis from vocal features and acoustic signals.Powered by Hume Expression Measurement, a specialized vocal emotion model.
Interruptions
Detailed interruption and overlap analysis from speaker diarization.Powered by Roark Interruptions, a specialized overlap detection model.
Quality
Experience quality scoring from conversational signals.Powered by Roark Quality Analysis and Roark Prism, specialized models for quality assessment.
Repetition Detection
Conversational loop and repetition analysis.Powered by Roark Prism, our evaluation model optimized for voice AI.
Tool Invocations
Analysis of function/tool calling behavior during conversations.Powered by Roark Vibe and Roark Prism.
Compliance
Regulatory and safety evaluation metrics for AI agent conversations.Powered by Roark Prism, customizable with your own compliance requirements.
Voicemail Detection
Voicemail detection and handling quality assessment.Powered by Roark Prism.
Call Screening
Detects when a screener answers an outbound call instead of the person being called: a receptionist, an assistant, a household gatekeeper, or an automated screen.Powered by Roark Prism.
Accent Detection
English accent identification from audio signals.Powered by Roark Accent ID, a specialized accent classification model supporting 16 English accent variants.
For a detailed walkthrough on using accent metrics, see the Accent Detection recipe.
Call Quality (DNSMOS)
Speech quality assessment using the ITU-T P.808/P.835 Mean Opinion Score (MOS) scale.Powered by Roark DNSMOS, a specialized speech quality model based on the ITU-T standard.
Telephony
What the phone carrier measured on a simulated call’s line. Read from the carrier’s own report of the call, not from the recording, so they say how the line carried the audio while Call Quality (DNSMOS) says how the audio sounds.Simulated phone calls only. A simulation that didn’t go over the phone network is marked not applicable, and a phone call with no carrier report is marked as missing data. See Phone network conditions.
Voice Naturalness
Acoustic naturalness of the agent’s synthesized voice.Powered by Roark UTMOS, a specialized naturalness model producing a mean opinion score.
Voice Human-Likeness
Perceptual human-likeness judged from the agent’s own audio.Powered by an audio-input evaluation model listening to the agent’s actual speech.
Call Environment
Zero-shot acoustic classification of the caller’s environment.Powered by a specialized audio classification model.
Pronunciation Analysis
Per-word pronunciation checking against expected pronunciations you configure.Powered by Roark Phoneme, a wav2vec2-based phoneme recognition model. Configure the word list and strictness per project.
Property Verification
Checks the call properties you send at ingest (customer name, account number, appointment time, and so on) against what was actually said on the call, and flags values that conflict. Useful for catching stale CRM data, transcription errors, and callers who are not who the metadata says they are.Powered by Roark Prism. Requires custom properties on the call; system-generated properties are never checked.
Every property is judged with a three-way verdict:
- MATCH: the conversation referred to this property and agrees with the value you sent, allowing for formatting differences, nicknames, partial references, and transcription noise.
- MISMATCH: the conversation states a genuinely different value.
- NOT_MENTIONED: the subject never came up. This is the most common outcome and is not counted as a mismatch.
propertyVerdicts array on the metric value: each entry carries the property name, the expected value, the verdict, the observed value for mismatches, the judge’s reasoning, and the transcript segment where the property was referred to. The same breakdown renders on the call detail page in the platform.
Records about the call rather than things anyone says in it, such as an outcome label (failed, timed_out), an internal id or a timestamp, come back as NOT_MENTIONED. A brand name and the legal name of the same organization (“Chase” and “JPMorgan Chase Bank, N.A.”) count as a MATCH.
Choosing which properties to verify
By default every custom property on the call is checked. To check only the values the call is expected to state, pick them under Properties to verify on the metric in a collector, run plan or flow. The other properties are then ignored, so metadata can never trip the mismatch flag. The three Property Verification metrics share one check, so a list set on any of them applies to all. The same list can be set in config as code:settings: { verifiedProperties: [...] } on the metric in POST/PUT /v1/metric/policies.
A changed list applies to calls scored afterwards. To re-score earlier calls with it, run a collection job with the collector’s id.
What’s Next
Custom Metrics
Create LLM Judge, Pattern, Formula, and API check metrics
Studio
Test metrics interactively against real calls
Collectors
Automate metric collection with conditions-based rules
Thresholds
Define pass/fail criteria for your metrics