> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Generate Simulation Resources

> Let Roark work out what to test for an agent, build the flows and checks, and keep them up to date as the agent and its calls change

## Overview

Roark can generate the simulation resources for an agent: the customer flows, the checks that grade them, and a run plan that runs them. It reads what you tell it about the agent and what the agent does on its calls, decides what is worth testing, and builds it. You review it and start a run when you are ready.

Every decision comes with a reason. Each flow and check it creates shows why it exists and, where it came from your material, the exact words in your prompt, your website or a call that led to it. Anything it chose to leave out is listed with the reason too.

Generation never runs your tests on its own. When it finishes, you get a run plan to start, and a webhook if you subscribed to one.

Run it again whenever you like. Each run builds on what is already in place: it reads what changed in the agent and its recent production calls, and adds only what is missing. When nothing has changed, it says so and adds nothing.

## When it runs

* **Creating an agent in the dashboard.** "Automatically set up simulations for this agent" is on by default on the new agent page. Open **Customize** to add testing instructions, choose capabilities or change the exploration limits.
* **Connecting a voice platform.** Turn on "Automatically set up simulations" when you connect, and Roark generates resources for each agent the connection brings in, including ones added later. It is on by default for connections with fewer than five agents.
* **Any time later.** From an agent's page, choose **Set up simulations** the first time, or **Generate simulation resources** after that. Through the API, call `POST /v1/simulation/generations/resources`.

Creating an agent through the API does not start a generation. Create the agent, then call the generate endpoint.

## What you give it

| Input | What it is for |
| - | - |
| Name and description | What the agent is for |
| System prompt | The agent's instructions: the main source for what it does and the rules it follows |
| Industry | Adds the rules and checks agents in that industry are expected to meet. Choosing **Other** means you pick the capabilities yourself |
| Website | A page about the business, for services, hours and policies the prompt does not mention. A later run without one reuses the last |
| Testing instructions | What you want tested, in your own words. Each scenario you describe becomes its own flow |
| Capabilities | What the agent does (for example booking, payments, call routing). Leave empty and Roark works them out |
| Exploration | Whether Roark calls the agent to learn what it actually does, how many calls it may place, and how long each call may last |

The more you give it, the closer the test calls are to your real callers. A name and a phone number are enough to start: Roark works from what the agent says on calls.

## What it builds

Generation works through a checklist. It does not finish while a part that applies to your agent is still uncovered.

| Part | What it creates |
| - | - |
| **Every agent** | Calls that try to make the agent misbehave (prompt injection, attempts to extract its instructions, impersonation, off-topic derailment), and checks on response time, silences, interruptions and how calls end |
| **Your industry** | The industry's rules, added to every flow, and its checks. For example, a healthcare agent is checked on how it handles personal information and on not giving medical advice |
| **What it does** | One flow per capability, built from Roark's tested scenarios for it and rewritten with your business's details, with the checks that measure it |
| **Your prompt and website** | Flows for situations your material describes that the capability flows do not cover, and checks for rules the agent must follow, such as verifying a caller before changing an appointment |
| **Your instructions** | A flow for each scenario you asked for |
| **Calling the agent** | Flows for paths Roark found on its calls, including ones your prompt never mentions, and checks for problems it saw, such as long silences |
| **Your production calls** | Flows for paths your customers take on the agent's real calls that nothing already tests, and checks for problems those calls show |

A part can be marked **Not needed** when there is nothing to build it from, for example when there are no testing instructions or no number to call. A part can also be skipped with a stated reason when it cannot be tested.

### Exploration calls

With exploration on, Roark places a few short calls to the agent, starting with a broad "what can you help with?" and then following the paths it offers. Exploration callers never confirm a booking, payment or other change: they stop just before the agent would commit it.

When the agent asks for something Roark does not have, such as an account number, the flow for that path is built but left out of the run plan, and the test data it needs is listed under **Test data Roark needs**. Add it, then include the flow.

Exploration needs a phone number Roark can call, or an outbound dial setup. It stops at your call limit, and after about two hours of exploring at most.

### Production calls

Roark reads the agent's newest production calls, up to 30: the ones since the last generation, or from the last 30 days the first time. It finds the paths customers take and the problems they run into. A path that an existing flow already tests is left out, with the reason. A new one becomes a flow. Open issues on the agent's calls are taken into account too.

## Running it again

Every run starts from what earlier runs built that still exists. Those flows and checks count as covered, so they are never built a second time.

A run looks at what happened since the last one:

* **The agent changed**: a new prompt, description or website, new testing instructions, a new industry or capabilities, or Roark can now call the agent.
* **New production calls** since the last run.
* **New activity on open issues** on the agent's calls.

When none of these happened, the run finishes within seconds, says there is nothing to add, and keeps the run plan as it is. When something did, it adds only what the change calls for. New flows and checks go into the existing run plan. A flow or check you took out of the plan stays out.

When the agent has not changed, Roark does not call it again: its production calls already show what it does.

## Following and guiding it

While it runs, the agent's page shows what it is doing in plain sentences. Open it to see:

* **The checklist**, with where each part stands.
* **Activity**: what it decided and did, in order.
* **Guide the generation**: tell it what to focus on or avoid, for example "focus on billing questions" or "skip transfers". It reads your guidance before its next step, and your guidance outranks its own plan.
* **Questions**: occasionally it asks something only you can answer. Each question has a default, and if nobody answers within 15 minutes it goes ahead with it, so a run nobody is watching still finishes.
* **Stop**: stops the run. Flows and checks it already built are kept, and the run plan is not made or changed.

When it finishes, **See why** opens the full list of what it built, grouped by part, with the reason and source for each item. A generated flow also shows its reason at the top of the flow.

## In the API

### Start a generation

```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
curl -X POST https://api.roark.ai/v1/simulation/generations/resources \
  -H "Authorization: Bearer $ROARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agentIds": ["3f2b1c9d-8e7a-4b6c-9d5e-1a2b3c4d5e6f"],
    "industry": "HEALTHCARE",
    "testingInstructions": "Test a caller who wants to cancel tomorrow and is upset about the fee.",
    "explore": { "maxConversations": 6, "maxConversationMinutes": 4 }
  }'
```

`agentIds` takes one agent for now. The response is the generation, with `status: "PENDING"`. A second request for the same agent while one is running returns `409`.

### Follow it

`GET /v1/simulation/generations/resources/{generationId}` returns one generation, and `GET /v1/simulation/generations/resources?agentIds=...` lists an agent's generations, newest first. Each has:

* `status`: `PENDING`, `RUNNING`, `NEEDS_INPUT` (waiting on a question), `COMPLETED`, `FAILED` or `CANCELLED`
* `layers`: the checklist, each part `COVERED`, `SKIPPED`, `OPEN` or `NOT_APPLICABLE`
* `activity`: what it did and said, in order
* `question`: the open question while `NEEDS_INPUT`, with its options and default
* `items`: every flow, metric and check this run created or left out, each with its `reason` and `source` (type, quote, page URL, or call and moment)
* `runPlanId`: the run plan to start once it completes
* `testDataNeeded`: data the agent asked for that Roark did not have

A run that found nothing to add completes with no `items` and keeps the earlier `runPlanId`.

Rather than polling, subscribe to the `simulation.resource_generation.completed` and `simulation.resource_generation.failed` webhooks. The completed event carries the `runPlanId`.

### Guide, answer or stop it

| Endpoint | Body |
| - | - |
| `POST /v1/simulation/generations/resources/{generationId}/guidance` | `{ "text": "Focus on billing questions" }` |
| `POST /v1/simulation/generations/resources/{generationId}/answer` | `{ "text": "Yes" }`, only while `NEEDS_INPUT` |
| `POST /v1/simulation/generations/resources/{generationId}/cancel` | none |

Each returns `409` once the generation has finished.

### Permissions

Starting, guiding, answering and stopping need an API key with `agent:update`, `simulation:create` and `metric:create`. Exploration places calls, so it also needs `simulation:run`: a key without it runs without exploration, unless you asked for exploration explicitly, in which case the request returns `403`. Reading generations needs `agent:read`.

## Billing

* **Exploration calls** are billed like any simulation call.
* **Generation's own work** (reading your material and calls, deciding what to test and reviewing calls) is billed as **AI agent usage**, one line per step. You can see it on your billing page. A run that stops because nothing changed costs nothing.

> [!TIP]
>
> Testing instructions are the quickest way to get the scenarios you care about most. Describe them the way you would brief a tester: who is calling, what they want, and what should happen.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.