Chat Simulations

​

Overview

Chat simulations run persona-driven conversations against text-based AI agents — the same testing primitives you use for voice, applied to text. Roark plays the human side of the chat, sends messages to your agent, and evaluates the resulting transcript against your metrics and checks. If you’ve run voice simulations, you already know how chat simulations work. The mental model is identical: a persona drives the customer’s behavior, a customer flow defines the conversation, a plan ties them together with an agent endpoint, and metrics measure the outcome.

---

​

Voice vs. Chat Endpoints

Whether a simulation runs as a call or a chat is decided by the endpoint type on the agent target — you don’t pick a modality separately.

---

​

How It’s Different from Voice Simulations

Chat simulations reuse every concept from voice simulations, with a few practical differences: Voice-only step types in Scripted flows (Silence, DTMF, and Voicemail steps in older flows) are skipped automatically when a flow runs against a chat endpoint, so you can reuse existing flows without forking them, and text-relevant steps still execute. Voicemail testing is voice-only for the same reason: a greeting is audio, so there’s nothing for a chat transcript to carry.

---

​

What’s Shared with Voice Simulations

Everything except the transport layer is shared:

• Personas — The attributes that shape text behavior carry over: language, base emotion, clarity, response timing, and disfluencies (natural false starts, rendered as typed hesitations). Voice-only attributes — accent, gender, speech pace — are simply ignored in a chat.
• Customer flows — Both Improv and Scripted flows work for chat. The same variants, agent expectations, and {{variable}} values apply.
• Plans — Build a chat plan the same way you build a voice one; the only difference is selecting an agent target with a chat endpoint.
• Variables — Variables resolve identically.
• Metrics and checks — System and custom metrics run on chat transcripts, and Pass/Fail checks evaluate the same way. Metrics that only apply to audio (e.g. speech rate, silence duration) are skipped.
• Schedules — Chat plans can be scheduled the same way as voice plans.
• Re-runs and reporting — Identical experience, with chat results appearing alongside call results in run reports.

Write your flows and personas once, then point them at either a voice endpoint or a chat endpoint depending on what you want to test.

---

​

How Chat Simulations Run

---

​

Watching Chat Simulations Live

While a run is in progress, chat sessions appear in the run’s Live now board just like calls. The live transcript and conversation events stream in real time as the persona and your agent exchange messages — the only difference from voice is that there’s no audio: the listen-in audio card is hidden for chat sessions.

---

​

Supported Providers

Chat simulations require a chat-capable agent endpoint. Provider endpoints are created automatically when you connect a supported integration — you don’t need to wire them up manually. You can also target a plain WebSocket endpoint for custom text agents. See each provider’s page for setup steps and required credentials.

---

​

Viewing Chat Results

Completed chat simulations appear in Simulate → Runs alongside call results, under the run’s SR- label. Opening a chat shows the full message transcript, persona context, and evaluated metric and check results. Chats are first-class conversation records in Roark — they show up in reports, dashboards, and collectors the same way calls do.

---

​

Next Steps