Skip to main content

​
Overview

By default, Roark captures the recording from its own simulation agent’s side of the conversation. This gives you basic transcript and metric analysis, but it’s limited to what Roark can observe externally. When you also send your agent’s call data to Roark — via an integration or the API/SDK — Roark matches and merges the two calls. This unlocks significantly richer analysis because your call carries data that only your agent has access to.

​
What You Gain

Enriched simulations give you access to data from your agent’s perspective, enabling deeper analysis:
DataDescription
Full TranscriptYour agent’s transcript, which may be higher quality or include internal annotations
Tool InvocationsThe actual tool calls your agent made during the conversation (e.g., database lookups, booking actions)
OpenTelemetry TracesBackend traces showing your agent’s internal processing
Properties & MetadataAny custom properties or context your agent attaches to the call
This additional context enables metrics that aren’t possible with the simulation recording alone. For example, the Tool Invocation metric can verify whether your agent actually called the right tool with the correct parameters — not just whether it said it would.

​
Enabling Call Merging in a Run

Call merging is controlled per run. In the create-run Metrics block, turn on Collect selected metrics from your agent’s side of the call. When the toggle is on, Roark waits for your agent’s copy of each call and scores the selected metrics against it.

​
Per-Metric Conversation Source

With the toggle on, each selected metric has a conversation source that determines which side of the call it’s scored against:
SourceScored againstBest for
SimulatedRoark’s side of the call — the simulation agent’s own recording and transcriptMetrics that only need what’s audible in the conversation
LiveYour agent’s side of the call — the call data your agent sends to RoarkMetrics that need internal data your agent carries

​
Live-Only Metrics

Some metrics can only be evaluated from your agent’s side of the call — tool invocations are the canonical example, since only your agent knows which tools it actually called. Selecting a live-only metric locks the call-merging toggle on: you can’t disable it while that metric is in the run.

​
Waiting and Fallback

When call merging is enabled, Roark waits up to 15 minutes for your agent’s side of each call to arrive. If it doesn’t show up in that window, Roark falls back and completes analysis with the simulated side only — live-sourced metrics won’t have your agent’s data to score against.
Send calls to Roark as soon as they end. The sooner your agent’s side arrives, the sooner merged results appear in the run report — and you avoid hitting the 15-minute fallback.
The API field behind the toggle keeps its internal name, enrichWithLiveConversation.

​
How Matching Works

Once your agent’s call arrives, Roark matches it to the simulation using two key signals:

​
Phone Number

The phone number your agent interacted with is matched against the number Roark provisioned for the simulation. This is the same dynamically assigned number described in Identifying Simulations.

​
Timing

The call must have started during the simulation’s active window. Roark checks that your call’s start time falls within the time range when the simulation phone number was in use for that specific test case. Both signals must match for the calls to be merged. This ensures accuracy even when phone numbers are reused across different runs.
Matching is bidirectional — it doesn’t matter which call arrives first. If your agent’s call arrives before Roark finishes processing the simulation, or vice versa, the merge happens once both are available (within the wait window described above).

​
Setup

Beyond the run toggle, the only requirement is that your agent already sends calls to Roark. If you haven’t set up call ingestion yet:
  1. Choose your method — Use a voice platform integration or send calls via the API/SDK
  2. Ensure calls include the phone number — Your call data must include the phone number that participated in the simulation so Roark can match it
  3. Enable the toggle when creating a run — Turn on Collect selected metrics from your agent’s side of the call in the Metrics block and pick the source for each metric
If you’re using a voice platform integration like Vapi, Retell, or LiveKit, calls are typically sent to Roark automatically — meaning enriched simulations work out of the box once you enable the toggle.

​
Best Practices

To get the most out of enriched simulations, send tool invocations with your call data. This enables live-sourced metrics that verify whether your agent executed the correct actions, not just whether it generated the right words.
Roark waits up to 15 minutes for your agent’s side before falling back to the simulated recording. Sending calls shortly after they end keeps live-sourced metrics complete and results fast.
Make sure the phone number in your call data matches the number your agent actually used during the simulation. Mismatched or reformatted numbers will prevent matching.
If your agent supports OpenTelemetry tracing, include trace data with your calls. This gives you full visibility into your agent’s internal decision-making during simulations.

​
Next Steps

Tool Invocations

Learn how to submit tool calls with your call data

Traces

Set up OpenTelemetry tracing for deeper analysis

Integrations

Connect your voice platform for automatic call ingestion

Identifying Simulations

Understand how simulation calls are identified