Chat Simulations

Documentation Index

Fetch the complete documentation index at: https://docs.roark.ai/llms.txt

Use this file to discover all available pages before exploring further.

​

Overview

Chat simulations let you run scripted, persona-driven conversations against text-based AI agents — the same testing primitives you use for voice, applied to text. Roark plays the human side of the chat, sends messages to your agent, and evaluates the resulting transcript against your metrics. If you’ve used call simulations, you already know how chat simulations work. The mental model is identical: a persona drives the customer’s behavior, a scenario defines the conversation, a run plan ties them together with an agent endpoint, and metrics measure the outcome.

---

​

How It’s Different from Call Simulations

Chat simulations reuse every concept from call simulations, with three practical differences: Voice-only step types (CUSTOMER_SILENCE, CUSTOMER_DTMF, VOICEMAIL) are filtered out automatically when a scenario runs against a chat endpoint, so you can reuse existing scenarios without having to fork them — text-relevant steps still execute, voice-only ones are skipped.

---

​

What’s Shared with Call Simulations

Everything except the transport layer is shared. Specifically:

• Personas — Every persona attribute that affects text behavior carries over: language, secondary language and code-switching, base emotion, intent clarity, confirmation style, memory reliability, disfluencies, backstory, custom properties. Voice-only attributes (voice ID, accent, speaking pace) are simply ignored when the persona is used in a chat simulation.
• Scenarios — The same scenario step structure, branch points, and template variables work for both modalities.
• Run plans — Build a chat run plan the same way you build a voice one; the only difference is selecting a chat-capable agent endpoint.
• Variables — Template variables resolve identically.
• Metrics & policies — System and custom metrics run on chat transcripts. Metric definitions that declare support for the chat conversation type are eligible; voice-specific metrics (e.g. speech-rate, silence-duration) are skipped.
• Schedules — Chat run plans can be scheduled the same way as voice run plans.
• Re-runs and reporting — Identical UX, with chat results appearing alongside call results in run summaries.

In other words: write your scenarios and personas once, then point them at either a voice endpoint or a chat endpoint depending on what you want to test.

---

​

How Chat Simulations Run

---

​

Supported Providers

Chat simulations require a chat-capable agent endpoint. The endpoint is created automatically when you connect a supported integration — you don’t need to wire it up manually. Each provider page documents the exact setup steps, required credentials, and any provider-specific behavior. At a high level:

• ElevenLabs — Reuses the same integration as voice. Once connected, you can run chat simulations against any synced agent using its WebSocket conversation endpoint.
• Google CES — Chat-only integration. Authenticates with Workload Identity Federation; Roark mints a short-lived access token at simulation time. No live monitoring or voice simulations.
• Kore AI — Chat-only integration. Uses an x-api-key and app coordinates (App ID, Environment, optional Base URL). No live monitoring or voice simulations.

---

​

Viewing Chat Results

Completed chat simulations appear in Simulations → Runs alongside call results. Clicking into a chat opens the conversation transcript view with the full message history, persona context, scenario step alignment, and any evaluated metric values. Chats are first-class conversation records in Roark — they show up in reports, dashboards, and metric collection jobs the same way calls do.

---

​

Next Steps