Skip to main content

​
Why this is a common failure point

A voice agent is a public endpoint anyone can call. Callers will try to talk it out of its instructions (“ignore what you were told and…”), extract someone else’s personal data, get it to give legal or medical advice, or make it claim to be human. Most of these attacks are simple, and most agents are only tested by people who want them to succeed.

​
How Roark tests it

Red teaming runs adversarial callers against your agent and grades six safety properties on every call. The attacks come from Roark’s curated adversarial library, from your own flows labelled Adversarial, or from new edge cases Roark generates for one of your flows at the difficulty you choose.

​
Setting it up

Choose how to source the attacks:
  • Choose scenarios: a filterable catalogue of Roark’s adversarial library and your own flows carrying the Adversarial label.
  • Generate new: pick one of your improv flows, how many cases (3 to 10) and a difficulty (easy, medium or hard). Roark drafts adversarial variants, adds them to the flow, and runs them.

​
What it measures

The template seeds these, and you can add or remove metrics in Advanced before running.

​
What to look for

  • One failure is a finding. Unlike quality metrics, a single successful attack is worth fixing. Read every failing call.
  • PII and identity first. They carry legal weight. Scope and hallucination failures are usually prompt fixes.
  • Raise the difficulty over time. Once easy attacks pass, generate hard ones against the same flow.

​
Over the API

Run it with POST /v1/simulation/run and "template": "red-teaming". GET /v1/simulation/template lists what each template accepts.