Welcome to Roark

Roark is where voice and chat agents prove they’re ready: simulate every failure mode before launch, analyze every production call once you’re live, and score them all with metrics powered by Roark Prism, our purpose-built evaluation model.

Simulation testing

Break it in staging, not in production. Test agents with synthetic callers across flows, personas, accents, and edge cases before you ship.

Post-call analysis

Every production call scored, filed, and traced. Transcribed and measured against your metrics in real time.

Human review

Label calls, set ground truth, and align every metric with your team’s judgment.

Autoimprove

From caught issue to drafted fix to proven deploy, with you in the loop.

Your first simulation

The fastest way in is a few lines of code: define an agent and a simulation plan, apply them, and run it from the CLI. It’s versioned in git, reusable in CI, and there are no UUIDs to wrangle.

import { defineAgent, defineSimulationPlan, defineConfig } from '@roarkanalytics/sdk/lib/config';

const frontdesk = defineAgent({
  name: 'frontdesk',
  endpoints: [{ value: '+15551234567', direction: 'OUTGOING' }],
});

export default defineConfig({
  resources: [
    frontdesk,
    defineSimulationPlan({
      name: 'smoke-test',
      direction: 'OUTBOUND',
      agentEndpoints: [{ agent: 'frontdesk', value: '+15551234567' }],
      flows: [{ system: 'sf-screener-apple' }], // a built-in flow; swap for your own
      metrics: ['call_screening_handling_score'],
      maxDurationSeconds: 300,
    }),
  ],
});

# agents/frontdesk.yaml
kind: agent
name: frontdesk
endpoints:
  - value: '+15551234567'
    direction: OUTGOING
---
# simulation-plans/smoke-test.yaml
kind: simulationPlan
name: smoke-test
direction: OUTBOUND
agentEndpoints:
  - agent: frontdesk
    value: '+15551234567'
flows:
  - system: sf-screener-apple
metrics:
  - call_screening_handling_score
maxDurationSeconds: 300

export ROARK_API_BEARER_TOKEN="<key>"        # needs config:apply + simulation permissions
roark config apply roark.config.ts           # create the agent + plan (or `roark config apply ./roark` for YAML)
roark simulation run --data '{"planId":"<plan-id>"}' # run it (find the id with `roark simulation plan list`)

Full walkthrough: Run a simulation

Define, apply, run, read the pass/fail, and gate CI on it, step by step.

Other ways to run one

Prefer to click through it, or drive the API imperatively? Each tab below also takes you from nothing to a running simulation, reusing a built-in persona and environment.

Node SDK
Python SDK
CLI
Config as Code
Dashboard

Copy, paste, run with @roarkanalytics/sdk:

import Roark from '@roarkanalytics/sdk'

const client = new Roark({ bearerToken: process.env.ROARK_API_BEARER_TOKEN })

// An agent and how to reach it
const agent = await client.agent.create({ name: 'Front Desk' })
const endpoint = await client.agentEndpoint.create({
  agentId: agent.data.id,
  value: '+15551234567',
  direction: 'INCOMING_AND_OUTGOING',
})

// Reuse a built-in persona and the "Quiet line" environment
const personas = await client.simulationPersona.list({ limit: 50 })
const envs = await client.simulationEnvironment.list({ limit: 50 })
const quietLine = envs.data.find((e) => e.name === 'Quiet line')!

// The conversation to test
const flow = await client.customerFlow.create({
  type: 'IMPROV',
  title: 'Rebooking',
  agentIds: [agent.data.id],
  happyPath: {
    title: 'Frustrated rebooking',
    personaOverrideId: personas.data[0].id,
    environmentId: quietLine.id,
    prompt: 'You call to rebook a cleaning that was cancelled on you.',
  },
  agentExpectations: [{ prompt: 'Agent offers a concrete alternative appointment time' }],
})

// Run it
const run = await client.simulation.run({
  plan: {
    direction: 'INBOUND',
    maxSimulationDurationSeconds: 300,
    agentEndpoints: [{ id: endpoint.data.id }],
    flows: [{ id: flow.data.id, happyPath: true }],
    metrics: [{ slug: 'task_completion' }],
  },
})

console.log(run.data.simulationRunPlanJobId)

Watch it in the dashboard, or poll client.simulationRunPlanJob.getByID(run.data.simulationRunPlanJobId).

Same flow with roark_analytics:

import os
from roark_analytics import Roark

client = Roark(bearer_token=os.environ["ROARK_API_BEARER_TOKEN"])

# An agent and how to reach it
agent = client.agent.create(name="Front Desk")
endpoint = client.agent_endpoint.create(
    agent_id=agent.data.id,
    value="+15551234567",
    direction="INCOMING_AND_OUTGOING",
)

# Reuse a built-in persona and the "Quiet line" environment
personas = client.simulation_persona.list(limit=50)
envs = client.simulation_environment.list(limit=50)
quiet_line = next(e for e in envs.data if e.name == "Quiet line")

# The conversation to test
flow = client.customer_flow.create(
    type="IMPROV",
    title="Rebooking",
    agent_ids=[agent.data.id],
    happy_path={
        "title": "Frustrated rebooking",
        "personaOverrideId": personas.data[0].id,
        "environmentId": quiet_line.id,
        "prompt": "You call to rebook a cleaning that was cancelled on you.",
    },
    agent_expectations=[{"prompt": "Agent offers a concrete alternative appointment time"}],
)

# Run it
run = client.simulation.run(
    plan={
        "direction": "INBOUND",
        "maxSimulationDurationSeconds": 300,
        "agentEndpoints": [{"id": endpoint.data.id}],
        "flows": [{"id": flow.data.id, "happyPath": True}],
        "metrics": [{"slug": "task_completion"}],
    },
)

print(run.data.simulation_run_plan_job_id)

Drive the same steps from your terminal (install the CLI and set ROARK_API_BEARER_TOKEN). Commands print JSON, so pipe through jq:

AGENT_ID=$(roark agent create --name "Front Desk" | jq -r '.data.id')

ENDPOINT_ID=$(roark agent endpoint create --agent-id "$AGENT_ID" \
  --value "+15551234567" --direction INCOMING_AND_OUTGOING | jq -r '.data.id')

PERSONA_ID=$(roark simulation persona list --limit 50 | jq -r '.data[0].id')
ENV_ID=$(roark simulation environment list --limit 50 \
  | jq -r '.data[] | select(.name=="Quiet line") | .id')

FLOW_ID=$(roark customer-flow create --data "$(jq -nc \
  --arg a "$AGENT_ID" --arg p "$PERSONA_ID" --arg e "$ENV_ID" '{
    type:"IMPROV", title:"Rebooking", agentIds:[$a],
    happyPath:{ title:"Frustrated rebooking", personaOverrideId:$p, environmentId:$e,
      prompt:"You call to rebook a cleaning that was cancelled on you." }
  }')" | jq -r '.data.id')

roark simulation run --data "$(jq -nc --arg ep "$ENDPOINT_ID" --arg f "$FLOW_ID" '{
  plan:{ direction:"INBOUND", maxSimulationDurationSeconds:300,
    agentEndpoints:[{id:$ep}], flows:[{id:$f, happyPath:true}],
    metrics:[{slug:"task_completion"}] }
}')"

Keep your test suite in git as YAML and apply it with the CLI:

roark/flows/rebooking.yaml

kind: flow
type: improv
name: rebooking
agents: [frontdesk]
happyPath:
  persona: frustrated-caller
  environment: Quiet line
  prompt: You call to rebook a cleaning that was cancelled on you.
expectations:
  - Agent offers a concrete alternative appointment time

roark config apply ./roark

Config defines your resources; launch a run from the CLI, an SDK, or the dashboard. Full example (agents + personas + flows) and how to run it in CI: Config as Code and CI/CD.

Explore the docs

Simulations

Customer flows, personas, templates, plans, and schedules

Config as Code

Define agents, personas, flows, metrics, collectors, and alerts as YAML in git

Observability

Call history, traces, reports, and dashboards

Metrics

The metric library, Studio, collectors, and thresholds

Integrations

Vapi, Retell, ElevenLabs, Leaping, LiveKit, and custom

CLI

Drive Roark from your terminal and CI pipelines

SDKs

Node.js, Python, and MCP Server

API Reference

REST API documentation

Getting Started

Simulations

Observability

Metrics

Autoimprove

Recipes

Integrations

Enterprise

Resources

Simulation testing

Post-call analysis

Human review

Autoimprove

Your first simulation

Full walkthrough: Run a simulation

Other ways to run one

Explore the docs

Simulations

Config as Code

Observability

Metrics

Integrations

CLI

SDKs

API Reference

Simulation testing

Post-call analysis

Human review

Autoimprove

​Your first simulation

Full walkthrough: Run a simulation

​Other ways to run one

​Explore the docs

Simulations

Config as Code

Observability

Metrics

Integrations

CLI

SDKs

API Reference

Your first simulation

Other ways to run one

Explore the docs