Skip to main content

​
Overview

A live call report is the report Roark writes after a simulation run, for your production traffic. You pick the calls (an agent, narrowed by filters) and the bar they are held to (targets). When each period closes, Roark reads the period’s calls and writes a report you can open in Roark, download as a PDF, or have sent to email and Slack. Each report answers four questions:
  • Did we hit our targets? Every target with this period’s number, the period before, and whether it was met.
  • What changed? Call volume, how calls ended, and every check’s pass rate against the previous period.
  • What went wrong? Failure themes, grouped from the calls that failed a target or check, each with how many calls it affected, quoted transcript evidence, and links to the calls.
  • What should we do? Recommendations tied to the themes, in priority order.
Define a report in code, or build it in Roark under Monitor > Reports.

​
Create a report

Open Reports in the sidebar and click New report. The form walks through six steps; the rail on the left shows what each is set to, and you can jump between them.
1

Details

The report’s name (how it reads in the list, emails and Slack) and an optional description.
2

Calls

The agent whose calls it covers, or every agent in the project, and any filters that narrow them further: a call property, the modality (calls or chats), a dataset, or another metric’s value (for example, only calls where the task was completed). Simulations are never included.
3

Targets

Up to 10 numbers each period is held to, for example “Task completion pass rate at least 85%” or “p95 latency at most 1,200 ms”. A report with no targets still describes the period; targets are what turn it into a verdict.
4

Schedule

Daily, weekly or monthly, the timezone the periods are read in, and the local hour the report is sent.
5

Delivery

Who gets each report: every project member by email (the default), a list of addresses (people without a Roark account included, since the email carries the PDF), no email, and any Slack channels. You can also turn off transcript excerpts here if transcripts must not leave Roark.
6

Charts

Optionally, a dashboard whose charts the report embeds. Its widgets are read over the report’s own calls and period, so the charts always match the numbers around them.
Use Write the latest period now from the report’s menu to see a report straight away, rather than waiting for the period to close.

​
Targets

A target is a metric, an aggregation, and a bar: A threshold’s pass rate uses the threshold’s own condition. For any other metric, a pass-rate target also takes the condition a call passes on (for example, “sentiment at least 6”). A target is met when the period’s number is at least the bar (the default for pass rates) or at most it (the default for everything else, such as latency). Each target is also read per agent, so a report covering several agents shows which one is pulling the number down.

​
Schedule and periods

A report covers whole calendar periods in its timezone: a day, a Monday-to-Sunday week, or a calendar month. It is written once the period has closed, at the delivery hour on the following day. A weekly report set to 08:00 in America/Chicago covers Monday to Sunday and arrives on Monday at 08:00 Chicago time. Each report is compared with the period before it: a week with the previous week, a month with the previous month. A comparison is only drawn when both periods have enough calls to make it meaningful. Pause a report from its menu to stop new reports without deleting it. When you resume it, the next report covers the most recent closed period; the periods it was paused for are not written retroactively, but you can still write the latest one on demand.

​
Reading a report

When many calls fail, Roark reads a representative sample of them and weights each theme back to the full count, so “affects about 340 calls” means 340 of the period’s calls, not 340 of the sample. The appendix says how many calls were read. Use the period picker to open an earlier report. Every report also has a Download PDF; links in emails and Slack open the report they were sent for, even if the report definition is later deleted.

​
API and CLI

Everything in the form is available over the API, so a report can be created from a script or a deployment pipeline.
A written report’s document carries the same content as the page: the verdict, targets, themes with their evidence, recommendations, per-agent rows and checks, so you can feed it into your own tooling. See the API reference for every field.
A report defined in config as code is read-only in Roark and over the API: edits and deletes return 409. Anyone can still write the latest period on demand.