Run Plans

Overview

A plan is a reusable test suite. It bundles everything a simulation run needs — the agents to test, the customer flows to exercise, the metrics and Pass/Fail checks to grade against, and the run settings — so you can execute the same suite repeatedly and compare results over time. Every run starts from a plan. When you configure a run without saving it, Roark still creates a plan behind the scenes — it just stays hidden as a one-off run. Check Save as plan and the configuration becomes a named plan you can re-run anytime, schedule, and edit.

Plans live under Simulate → Simulations in the sidebar. The runs hub has a Plans / All runs view toggle — the Plans view shows one card per saved plan, plus a pinned One-off runs card for runs that weren’t saved as a plan.

Creating a plan

Click New Run on the Simulations hub to open the create page. It’s a single long-form page with four numbered sections:

01 · Agents to test

Pick the agents and how to reach each one. Select agents & endpoints opens a picker; each target is one agent reached on one specific endpoint — Phone, Web RTC, LiveKit, WebSocket, ElevenLabs, Google CES, or Kore — with its direction shown as Inbound, Outbound, or In + out.

02 · Template

Start from a goal. Each template ships with preset metrics, Pass/Fail checks, and a starter configuration. Pick from the card grid (Blank is the default) or open Browse all templates for the full library. See Templates for what each one does.

03 · Template configuration

The template’s own controls — for example, Load testing shows a Volume panel with concurrent calls and total iterations, and Multilingual asks which languages to test. Every template also surfaces a Flows panel where you attach the customer flows the run exercises (see below).

04 · Advanced

Metrics and run settings: the metric picker with Pass/Fail thresholds, the option to collect metrics from your agent’s side of the call, run settings, end conditions, and custom variables.

Save as plan, or run one-off

The footer bar has a Save as plan checkbox: “Saves this configuration as a plan you can re-run anytime.” Checking it prompts you to name the plan.

Checked — the plan appears in the Plans view with its own detail page, and every execution is recorded in its run history.
Unchecked — the run executes as a one-off. It’s grouped under the One-off runs card in the Plans view and the One-off lens in All runs. You can promote a one-off to a named plan later from the run header.

Run now, or later

The primary button is a split button. Its face reads Run simulation (or Schedule when a schedule mode is selected), and the menu offers:

Option	What it does
Run now	Starts the simulation immediately
Schedule recurring…	Runs the plan on a cadence — hourly, daily, weekly, or monthly
Run once, later…	Runs a single time at a future date and time

Choosing a schedule mode reveals the Schedule section: frequency, time, timezone, days, and an end condition (never, on a date, or after N runs). See Schedules for details.

If any variable in the plan lacks a default value, a runtime-variables modal collects the missing values before the run starts.

Direction is derived

You don’t choose inbound or outbound — Roark derives it from the endpoints you selected in Agents to test. A run is outbound only when every selected endpoint is outgoing-only; otherwise it’s inbound. Outbound runs are forced to a single iteration because your agent initiates each call. Learn more in the Inbound vs Outbound guide.

Attaching flows

The Attach flows dialog lists your customer flow library — pick the flows this run exercises, and expand one to choose its variants. Each attachment carries three things:

Variant selection mode

Per attached flow, choose which variants run:

All variants — the happy path plus every edge case
Default variant — just the happy path
Specific variants — an explicit selection of variants

Templates set this for you where it matters — Multilingual and Load testing pin the default variant, for example.

Persona override

Optionally override the persona the flow’s variants would normally use, just for this plan. The Multilingual template uses this to fan the same flow across language personas — one attachment per language.

Variable overrides

Set per-attachment values for the flow’s {{variable}} tokens. Overrides set here win over the values stored on the variant at run time. See Variables.

The total number of test calls is the selected variants across all attachments, multiplied by your agent targets and iterations.

Run settings

The Advanced section exposes the run’s execution and end-condition settings:

Setting	Description	Default
Iterations	Runs per test case, 1–100. Forced to 1 for outbound runs.	1
Concurrency	Parallel calls, capped by your account quota	5
Execution mode	Parallel, Sequential — this plan, or Sequential — all plans	Parallel
Max duration	Hard cap per simulation, up to 24 hours	15 minutes
Silence timeout	Ends the call after sustained silence, 5–300 seconds	30 seconds
End-call phrases	Phrases that end the call when spoken (max 10; empty disables)	`goodbye`
End-call reasons	Semantic conditions evaluated by an LLM that end the call, e.g. “The agent confirmed the order” (max 10; empty disables)	None

The sequential execution modes run calls one at a time — scoped either to this plan’s runs or to every plan in the project — which is useful when your agent can’t handle parallel traffic.

Run-plan variables

Plans can define their own variables — a name, a value type, and an optional default — referenced as {{name}} in flow prompts and settings. Variables without a default are collected in a modal each time the plan runs, which makes them handy for values that change per run, like a booking date or a ticket number.

The plan detail page

Every saved plan has a detail page with the plan name, its template badge, the agents it targets, and its run count, plus two header actions: Edit (reopens the create form pre-filled — the template is read-only) and Run now. The body shows:

Automation panel — if the plan is scheduled: an Active/Paused badge, the cadence description, next-run and last-run times, and controls to pause/resume, edit the cadence, or delete the schedule. Unscheduled plans show a Runs on demand card with a Schedule this plan button instead.
Run history — every execution of the plan, newest first. Runs are labelled SR-{n} and show one of five statuses: Running, Queued, Completed, Failed, or Cancelled, plus a trigger chip (Manual, Schedule, Re-run, or System). Click a run to open its report — see Running Simulations.

Creating plans via API

You can create and trigger plans programmatically:

curl -X POST "https://api.roark.ai/v1/simulation/plan" \
  -H "Authorization: Bearer $ROARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production regression suite",
    "direction": "INBOUND",
    "maxSimulationDurationSeconds": 900,
    "scenarios": [
      {
        "id": "flow-uuid",
        "variables": {
          "customerName": "John Doe",
          "appointmentDate": "2026-02-15"
        }
      }
    ],
    "personas": [{ "id": "persona-uuid" }],
    "agentEndpoints": [{ "id": "endpoint-uuid" }],
    "metrics": [{ "slug": "task-completion" }],
    "iterationCount": 1,
    "maxConcurrentJobs": 5,
    "silenceTimeoutSeconds": 30,
    "endCallPhrases": ["goodbye"],
    "endCallReasons": ["The agent confirmed the order"],
    "executionMode": "PARALLEL",
    "autoRun": true
  }'

Required fields: name, direction, maxSimulationDurationSeconds, scenarios, personas, agentEndpoints, and metrics. The same flow ID can appear in scenarios multiple times with different variables, and autoRun: true triggers a run immediately after creation. Trigger an existing plan with POST /v1/simulation/plan/{planId}/job.

The REST API keeps the older names: customer flows are attached via the scenarios array, and direction is set explicitly rather than derived from endpoints as it is in the UI.

Next steps

Customer Flows

Author the conversations your plans exercise

Templates

Start plans from a goal with preset metrics and checks

Schedules

Put plans on a cadence and catch regressions automatically

Running Simulations

Watch runs live and read the report when they settle

Getting Started

Observability

Metrics

Simulations

Recipes

Integrations

SDKs & Libraries

Resources

Overview

Creating a plan

Save as plan, or run one-off

Run now, or later

Direction is derived

Attaching flows

Run settings

Run-plan variables

The plan detail page

Creating plans via API

Next steps

Customer Flows

Templates

Schedules

Running Simulations

​Overview

​Creating a plan

​Save as plan, or run one-off

​Run now, or later

​Direction is derived

​Attaching flows

​Run settings

​Run-plan variables

​The plan detail page

​Creating plans via API

​Next steps

Customer Flows

Templates

Schedules

Running Simulations

Overview

Creating a plan

Save as plan, or run one-off

Run now, or later

Direction is derived

Attaching flows

Run settings

Run-plan variables

The plan detail page

Creating plans via API

Next steps