Skip to main content
Autoimprove is Roark working on your agent instead of just measuring it. You pick one objective, a metric that should pass and the pass rate you want, and Roark runs the whole improvement loop autonomously: it clones your agent into a staging copy, writes caller scenarios that exercise the objective, measures a baseline over real simulated phone calls, changes the staging configuration, and re-tests until the objective holds. Production changes exactly once, when you promote a verified job’s changes.

​
Quickstart

Three commands with the Roark CLI. Find your agent and the metric that defines success, then hand Roark the objective:
That is the whole job description. Roark takes it from there; watch it work with:
The response carries the live worklog: every step, every batch of test calls with its pass rate, and Roark’s own running notes. The same job is live in the dashboard under Autoimprove, where the worklog renders as a feed.
The objective metric should produce a pass/fail verdict per call: either a boolean metric, or a threshold on a scale metric (for example “Instruction Follow >= 4”). Create one in the dashboard’s metric studio or with roark metric definitions create.

​
How it works

  1. Stage. Roark clones your production agent inside your provider account (Vapi or Retell) as a clearly-marked shadow, gives it its own phone number, and registers it for simulations. Your production agent is never modified by the loop.
  2. Author. It writes caller scenarios for the objective: the failure case it is hunting, an easy case, an edge case. Each becomes a real simulation flow with success criteria, graded by your metrics.
  3. Measure. A baseline batch of simulated calls establishes where things stand. Every later change is measured against it.
  4. Change and re-test. Roark edits the staging configuration through an audited funnel, then proves or disproves each change with more calls. It reads the graders’ evidence, not its own opinion of the transcripts.
  5. Verify and hand over. When the objective passes its target with repeated trials, the job waits for you: review the diffs and the numbers, then promote or dismiss.

​
What Roark can change

The whole provider configuration document, not just the prompt: the system prompt, the underlying model and temperature, the voice, the transcriber, timeouts and endpointing, the first message, tool definitions. Credentials, URLs, webhooks, and privacy settings are protected and refused by the platform.

​
The safety model

Staging-only writes

The loop only ever writes to the staging copy. The one write path to production is the promote, and only you trigger it.

Rollbackable promotes

A snapshot of the production configuration is taken immediately before a promote, so every promote can be fully rolled back.

Guaranteed cleanup

Whatever the outcome, everything the job set up is removed: the shadow agent, its phone number, the authored test flows and run plan.

Hard budgets

Decision turns and simulated calls are capped (10 turns and 200 calls by default, tunable at create). Roark cannot exceed them.

​
Job statuses

​
Go deeper

Running a job

The full lifecycle over the CLI and API: creating, watching, steering, answering, promoting.

CLI

Install and authenticate the Roark CLI used throughout this section.