Templates

​

Overview

Templates are step 01 · Template of creating a run. Instead of assembling metrics and checks from scratch, you pick a goal (measure conversation quality, probe for jailbreaks, hammer one flow at volume) and the template seeds a sensible starting configuration:

• Preset system metrics: a curated set from the metric library, auto-selected in the Advanced section.
• Pass/Fail checks: thresholds attached alongside the metrics, so the run reports a clean pass rate out of the box.
• A configuration panel: step 03 of the create page changes per template (languages to test, volume to run at, adversarial cases to include).
• Flow sourcing: some templates pre-attach customer flows for you, by system label or by happy path.

Everything a template seeds is a starting point, not a lock. You can add or remove metrics, adjust checks, and change flows in the Advanced section before running.

---

​

The catalogue

Pick a template from the card grid. The grid shows the common set; Show more expands it in place to reveal the rest. Each template has its own page covering why it matters, what it runs, what it measures and what to look for.

---

​

Property sweeps

The seven Robustness templates are property sweeps. Each runs the same flow once per value of one caller property (every accent, every background, every pace) with everything else held the same, then tells you which values did worse than the rest and which differences are just chance. Their configuration panel picks the flow, the values, a baseline to compare against, and the iterations, which it holds at 5 or more calls per value so every value can be compared.

---

​

Template provenance

When you check Save as plan, the plan records which template it came from. The template shows up as a badge on the plan’s detail page and on every run’s header, and when you edit a plan the template is read-only: it defined the plan’s shape, so you adjust the configuration rather than swap the goal. One plan, one template: to test the same agents against a different goal, create a new run from the other template and save it as its own plan.

---

​

Next steps