Core Concepts
Agent
A voice AI system that handles conversations. Can be a chatbot, voice assistant, or any automated system that interacts with users through voice or text.
Call
A single conversation session between participants (human and/or AI agents). Contains transcript, audio, metadata, and analysis results.
Endpoint
The connection point Roark uses to reach your agent. Endpoint types include Phone, Web RTC, LiveKit, WebSocket, ElevenLabs, Google CES, and Kore. Endpoints also carry a direction — whether Roark dials your agent (outbound from Roark) or your agent calls into Roark (inbound).
Evaluation
The process of scoring and analyzing calls against specific criteria to measure quality and performance.
Session
A group of related calls, useful for tracking multi-call interactions or conversation threads with the same customer.
Simulation & Testing
Persona
A simulated customer profile with defined characteristics, behaviors, and communication patterns used when running customer flows.
Customer Flow
One type of customer conversation your agent should handle — the customer setup, its variants, and the agent expectations it’s graded against. Customer flows supersede scenarios and drive simulations today, with live-call grading planned.
Variant
One concrete version of a customer flow. Every flow has a default variant (the happy path) plus any number of edge cases, each with its own setup, variables, and optional additional expectations.
Happy Path
A flow’s default variant — the expected, unexceptional version of the conversation.
Edge Case
A non-default variant of a customer flow that tests a deviation from the happy path (an upset customer, a wrong account number, an unusual request).
Improv Mode
A flow authoring mode where you write a free-form customer brief and the simulated customer improvises the conversation around it.
Scripted Mode
A flow authoring mode where you build the conversation as a step-by-step graph, with branching paths the customer follows.
Agent Expectations
The graded contract attached to a customer flow — what your agent must do to pass. Flow-level expectations are inherited by every variant; edge cases can add additional expectations of their own.
Flow Link
A scripted step that splices another scripted flow’s steps in at that point, so shared segments (like identity verification) can be reused across flows.
Template
A pre-built starting point for a run — such as Flow adherence, Red teaming, Conversation quality, Multilingual, Load testing, or Tool call accuracy — that seeds flows, metrics, and settings you can customize.
Run Plan
A reusable, named test suite: the agent targets to test, the customer flows attached, the metrics and checks to evaluate, and run settings like iterations and concurrency. One-off runs are plans that simply aren’t saved (hidden plans).
Run
A single execution of a run plan, labelled SR-. Runs move through five statuses: Running, Queued, Completed, Failed, and Cancelled.
Simulation
A controlled test conversation that validates agent behavior without involving real customers.
Regression Testing
Re-running previous customer flows against updated agent versions to ensure improvements don’t break existing functionality.
Analytics & Metrics
Collector
The user-facing name for a rule that runs selected metrics on matching calls — which metrics to evaluate, and the conditions (agent, source, call property) a call must match. Collectors can be Active or Paused.
The REST API and SDKs still use the older name for collectors: endpoints live under /v1/metric/policies and SDK methods under client.metricPolicy.*.
Studio
The workspace for building and testing metrics. Author mode defines a new metric (LLM Judge, Pattern, or Formula); Evaluate mode runs a battery of evals against sample calls or a dataset.
Dataset
A curated set of calls used as an evaluation corpus — for example, to run a metric battery in Studio against a fixed, representative sample.
Sentiment Analysis
Detection and classification of emotions and feelings expressed during conversations (64+ emotions tracked).
Sentiment Score
A numerical value (-1 to +1) indicating overall emotional tone, where negative values indicate negative sentiment and positive values indicate positive sentiment.
Quantitative measurements of agent efficiency including response time, resolution rate, and conversation duration.
When an agent calls an external function or API during a conversation (e.g., checking order status, booking appointment).
Talk Ratio
The percentage of conversation time each participant speaks, used to measure conversation balance.
Integration Terms
Webhook
An HTTP callback that sends real-time event data from voice platforms to Roark when calls occur.
WebSocket
A persistent connection protocol used for real-time, bidirectional communication in live monitoring.
Ingestion
The process of receiving and processing call data from voice platforms into Roark’s analytics system.
Egress
Outbound data flow, commonly referring to call recordings being sent to storage systems.
VAPI
A voice AI platform that provides infrastructure for building and deploying voice agents.
Retell AI
A platform for creating conversational AI agents with natural voice interactions.
LiveKit
An open-source platform for real-time audio/video communications, used for voice agent infrastructure.
Pipecat Cloud
A cloud platform for deploying and managing voice AI pipelines.
Technical Components
Batch Evaluation
Processing multiple calls simultaneously for analysis rather than one at a time.
Custom Metric
A user-defined metric — typically an LLM evaluation prompt — tailored to specific business requirements beyond the built-in system metrics.
Transcript
The text representation of spoken conversation, including speaker labels and timestamps.
Utterance
A single continuous speech segment from one speaker, ending when they stop talking or another speaker begins.
Turn
A complete speaking opportunity for one participant in a conversation, may contain multiple utterances.
Monitoring & Reporting
Live Monitoring
Real-time observation and analysis of ongoing calls as they happen.
Dashboard
The visual interface displaying analytics, metrics, and insights about your voice AI performance.
Baseline
A reference point for normal performance metrics, used to identify deviations and improvements.
Alert
Automated notification triggered when specific conditions are met (e.g., high frustration detected).
Trend Analysis
Tracking metrics over time to identify patterns, improvements, or degradations in performance.
Business Terms
First Call Resolution (FCR)
Successfully resolving a customer’s issue during their first call without requiring follow-up.
Escalation
Transferring a call to a human agent or supervisor when the AI cannot handle the situation.
Quality Assurance (QA)
The process of monitoring and evaluating agent interactions to ensure quality standards are met.
Service Level Agreement (SLA)
Performance standards and metrics that must be maintained (e.g., response time, resolution rate).
API & Development
API Key
A unique identifier used to authenticate requests to Roark’s API endpoints.
Rate Limiting
Restrictions on the number of API requests allowed within a time period to prevent abuse.
SDK
Software Development Kit - pre-built libraries (Node.js, Python) that simplify integration with Roark.
REST API
The HTTP-based interface for programmatically interacting with Roark’s services.
Payload
The data package sent in API requests or webhook events containing call information.
