Overview

UI-first or code-first? Build your customer flows and personas (and the agents they target) in the dashboard, or define them as config as code - YAML in your git repo, applied with the CLI. Both manage the same resources; run plans and execution live in the UI and API.

Quickstart

Already have an agent and a customer flow? Run a simulation against them:

Node
Python
CLI

const run = await client.simulation.run({
  plan: {
    direction: 'INBOUND',
    maxSimulationDurationSeconds: 300,
    agentEndpoints: [{ id: '<endpoint-id>' }],
    flows: [{ id: '<flow-id>', happyPath: true }],
    metrics: [{ slug: 'task_completion' }],
  },
})
console.log(run.data.simulationRunPlanJobId)

run = client.simulation.run(plan={
    "direction": "INBOUND",
    "maxSimulationDurationSeconds": 300,
    "agentEndpoints": [{"id": "<endpoint-id>"}],
    "flows": [{"id": "<flow-id>", "happyPath": True}],
    "metrics": [{"slug": "task_completion"}],
})
print(run.data.simulation_run_plan_job_id)

roark simulation run --data '{
  "plan": {
    "direction": "INBOUND",
    "maxSimulationDurationSeconds": 300,
    "agentEndpoints": [{ "id": "<endpoint-id>" }],
    "flows": [{ "id": "<flow-id>", "happyPath": true }],
    "metrics": [{ "slug": "task_completion" }]
  }
}'

Starting from scratch? Follow the full zero-to-first-simulation quickstart.

What is simulation testing?

Simulation testing lets Roark act as your customers. You describe the conversations your agents should handle (who’s calling, what they want, how the call should go) and Roark places real calls (or chat sessions) against your agents, grades every conversation, and reports pass/fail results. Everything revolves around plans and runs. A plan is a reusable test suite; a run is one execution of it. Each run is labelled SR-{n} and moves through Queued → Running → Completed (or Failed / Cancelled). A plan composes four things:

Building block	What it defines
Agent targets	Which agents to test and how to reach each one: an agent plus an endpoint (Phone, WebRTC, LiveKit, WebSocket, ElevenLabs, Google CES, Kore) with a direction (inbound, outbound, or both)
Customer flows	The conversations to exercise: each flow attached with a variant selection (all variants, the happy path only, or specific variants)
Metrics & checks	What to measure on every conversation, with pass/fail thresholds that decide the run’s verdict
Run settings	Iterations, concurrency, execution mode, and end conditions like max duration and silence timeout

You’ll find everything under the Simulate section of the sidebar: Simulations (the runs hub), Customer flows, and Personas.

Core concepts

Customer flows

A customer flow describes one type of conversation your agent should handle. Author it in Improv mode (write a free-text customer brief and the simulator improvises a fresh conversation each run) or Scripted mode (author the conversation step by step on a graph canvas, ideal for IVR menus and deterministic routing). Every flow has a default variant (the happy path) plus optional edge cases that vary the persona, environment, or setup. The flow’s agent expectations are the graded contract: LLM-checked instructions the agent is evaluated against on every variant.

Personas

A persona is who’s calling: language, accent, gender, base emotion, and how they speak. Each flow variant pins a persona, and run plans can override it per attachment, for example, fanning one flow across language personas in a multilingual run.

Plans and runs

A plan is the saved suite; a run is one execution. Run a plan on demand, put it on a schedule, or skip saving entirely. Unchecked Save as plan gives you a one-off run. Templates like Red teaming, Multilingual, and Load testing pre-configure metrics, checks, and flow sourcing so you start from a goal instead of a blank page.

Metrics and checks

Every run scores its conversations with metrics: system metrics out of the box, plus custom metrics you build in Studio. Pass/fail thresholds turn metrics into checks, and the run report’s verdict is the pass rate across every check on every conversation.

How a run comes together

Pick agents to test

From Simulations → New Run, select agents and the endpoints to reach them on. Run direction is derived from your endpoints: inbound means Roark calls your agent; outbound means your agent calls Roark.

Choose a template

Start from a goal (Flow adherence, Red teaming, Conversation quality, Multilingual, Load testing, Tool call accuracy) or Blank. Each template ships preset metrics and pass/fail checks, then attach the customer flows the run should exercise.

Tune the details

The Advanced section covers the metric set, iterations and concurrency, and end conditions (max duration, silence timeout, end-call phrases). Check Save as plan to keep the configuration as a re-runnable suite.

Run and review

Run now or schedule it. While the run is live you can watch calls in progress and listen in; once it settles, the report shows the verdict, per-metric results, run-over-run comparison, and every conversation’s transcript.

The REST API and SDKs keep the older naming: customer flows are still scenarios in request fields, and run plans are managed via the same endpoints as before. Code examples throughout these docs reflect the API’s names.

Common use cases

Pre-deployment testing: exercise every flow’s happy path and edge cases before an agent goes live
Regression testing: save a plan, re-run it after every change, and compare runs side by side in the report’s Comparison tab
Continuous monitoring: schedule plans hourly, daily, or weekly to catch regressions before you do
Safety and robustness: the Red teaming template runs adversarial flows against your agent; Load testing checks behavior under concurrent call volume

Where to go next

Customer flows

Author the conversations your agents should handle: Improv briefs or Scripted graphs

Personas

Define the callers your simulations run as

Templates

Start a run from a goal with preset metrics and checks

Plans & schedules

Build reusable suites and run them on a cadence

Running simulations

Launch runs, monitor them live, and read the report

Best practices

Strategies for building an effective test suite

For chat-based agents, see Chat simulations. To score metrics from your agent’s side of the call, see Enriched simulations.

Getting Started

Simulations

Observability

Metrics

Config as Code

Recipes

Integrations

Enterprise

SDKs & Developer Tools

Resources

Quickstart

What is simulation testing?

Core concepts

Customer flows

Personas

Plans and runs

Metrics and checks

How a run comes together

Common use cases

Where to go next

Customer flows

Personas

Templates

Plans & schedules

Running simulations

Best practices

​Quickstart

​What is simulation testing?

​Core concepts

​Customer flows

​Personas

​Plans and runs

​Metrics and checks

​How a run comes together

​Common use cases

​Where to go next

Customer flows

Personas

Templates

Plans & schedules

Running simulations

Best practices

Quickstart

What is simulation testing?

Core concepts

Customer flows

Personas

Plans and runs

Metrics and checks

How a run comes together

Common use cases

Where to go next