Code-first agents (LiveKit, Pipecat)

A LiveKit or Pipecat agent is your code on your infrastructure, so there is no provider document for Roark to edit. The managed-config integration turns that around: your agent fetches its tunable surface (prompt, model, params, voice) from Roark at session start, and Autoimprove works on that config with the same loop, safety model, and promote gate as provider agents. The key mechanic is per-session resolution: Roark’s test calls dial your production deployment directly. Because Roark originated those calls, the config endpoint recognizes them and serves the staging revision for those sessions only. Real callers always get production config; you deploy nothing extra.

1. Fetch config at session start

One call registers the key on first use (your defaults become revision 1) and resolves the right revision per session.

import os
from livekit import agents
from livekit.agents import AgentSession
from livekit.plugins import openai
from roark_analytics import Roark

roark = Roark(bearer_token=os.environ["ROARK_API_BEARER_TOKEN"])

DEFAULTS = {
    "system_prompt": "You are a helpful scheduling assistant for Mary's Dental.",
    "model": "gpt-4.1",
    "temperature": 0.4,
    # Externalize your STT and TTS choices too: whatever lives in this
    # document, Roark can tune. Whatever stays in code, it cannot.
    "tts": {"voice_id": "alloy", "speed": 1.0},
    "stt": {"language": "en"},
}

async def entrypoint(ctx: agents.JobContext):
    await ctx.connect()
    participant = await ctx.wait_for_participant()

    # SIP attributes carry the numbers on telephony rooms; adapt to your setup.
    caller = participant.attributes.get("sip.phoneNumber")
    called = participant.attributes.get("sip.trunkPhoneNumber")

    cfg = roark.agent_config.resolve(
        "support-agent",
        defaults=DEFAULTS,
        session={"session_id": ctx.room.name, "caller_number": caller, "called_number": called},
    ).data
    doc = cfg.document

    # Stamp attribution so Roark ties this session to the right revision
    # (and, for its own test calls, to the simulation that placed them).
    await ctx.room.local_participant.set_attributes(
        {"roark.revisionId": cfg.revision_id, "roark.channel": cfg.channel}
    )

    session = AgentSession(
        llm=openai.LLM(model=doc["model"], temperature=doc["temperature"]),
    )
    await session.start(agent=agents.Agent(instructions=doc["system_prompt"]), room=ctx.room)

import os
from pipecat.services.openai.llm import OpenAILLMService
from roark_analytics import Roark

roark = Roark(bearer_token=os.environ["ROARK_API_BEARER_TOKEN"])

DEFAULTS = {
    "system_prompt": "You are a helpful scheduling assistant for Mary's Dental.",
    "model": "gpt-4.1",
    "temperature": 0.4,
    # Externalize your STT and TTS choices too: whatever lives in this
    # document, Roark can tune. Whatever stays in code, it cannot.
    "tts": {"voice_id": "alloy", "speed": 1.0},
    "stt": {"language": "en"},
}

async def run_bot(transport, call_info):
    # call_info: however your transport surfaces the dial-in numbers.
    cfg = roark.agent_config.resolve(
        "support-agent",
        defaults=DEFAULTS,
        session={
            "session_id": call_info.session_id,
            "caller_number": call_info.caller_number,
            "called_number": call_info.called_number,
        },
    ).data
    doc = cfg.document

    llm = OpenAILLMService(model=doc["model"], params={"temperature": doc["temperature"]})
    messages = [{"role": "system", "content": doc["system_prompt"]}]
    # ... build your pipeline as usual, and include cfg.revision_id and
    # cfg.channel in the call properties you report to Roark.

Cache the resolved config for the session; do not fetch per turn. Wrap the resolve in a try/except that falls back to your DEFAULTS: the failure mode is then your current behavior, never a broken call. The SDK method ships in the next roark_analytics release; until you upgrade, the same call is one POST /v1/agent-config/{key}/resolve request.

2. Link the config to your Roark agent

Autoimprove finds the config through the agent your calls already report. Link once:

roark agent-config update support-agent \
  --channel production \
  --document @config.json \
  --agent-id "$AGENT_ID"

curl -X PUT https://api.roark.ai/v1/agent-config/support-agent \
  -H "Authorization: Bearer $ROARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"channel": "production", "document": {"system_prompt": "...", "model": "gpt-4.1", "temperature": 0.4}, "agentId": "'"$AGENT_ID"'"}'

Your agent also needs a registered phone endpoint (the number Roark’s simulations dial): roark agent endpoint create if you have not already.

3. Run Autoimprove

Exactly the same as any other agent:

roark autoimprove job create \
  --agent-id "$AGENT_ID" \
  --objective-metric-definition-id "$METRIC_ID" \
  --objective-label "Booking completion should pass" \
  --target-value 90

What happens under the hood differs only in the staging story: instead of cloning a shadow, Roark forks a staging config channel from production. Its test calls dial your production number, get recognized, and read the staging revision; every change Roark makes is a new audited revision on that channel. Real traffic never sees any of it. When the job reaches AWAITING_PROMOTE, promote is a pointer swap: your very next session reads the verified revision. Rolling back is writing the prior revision back to production; the chain preserves every state.

roark autoimprove job promote <job-id>

What Roark can and cannot improve here

Roark can change anything in the document you externalized: prompt, model and sampling params, and your STT/TTS choices (voice, speed, language) when you include them, as the example does. It keeps the key shape your code reads (it never renames keys). What stays in code (tool logic, pipeline structure) is out of reach by design; the job’s final report recommends those changes for a human when the config surface is not enough.

Getting Started

Simulations

Observability

Metrics

Config as Code

Autoimprove

Recipes

Integrations

Enterprise

SDKs & Developer Tools

Resources

Code-first agents (LiveKit, Pipecat)

1. Fetch config at session start

2. Link the config to your Roark agent

3. Run Autoimprove

What Roark can and cannot improve here

​1. Fetch config at session start

​2. Link the config to your Roark agent

​3. Run Autoimprove

​What Roark can and cannot improve here

1. Fetch config at session start

2. Link the config to your Roark agent

3. Run Autoimprove

What Roark can and cannot improve here