# Create a new agent endpoint Source: https://docs.roark.ai/api-reference/agent-endpoint/create-a-new-agent-endpoint /api-reference/openapi.documented.json post /v1/agent/endpoint Creates a new agent endpoint for the authenticated project. # Get agent endpoint by ID Source: https://docs.roark.ai/api-reference/agent-endpoint/get-agent-endpoint-by-id /api-reference/openapi.documented.json get /v1/agent/endpoint/{endpointId} Returns a specific agent endpoint by its ID. # List agent endpoints Source: https://docs.roark.ai/api-reference/agent-endpoint/list-agent-endpoints /api-reference/openapi.documented.json get /v1/agent/endpoint Returns a paginated list of agent endpoints for the authenticated project. # Update an agent endpoint Source: https://docs.roark.ai/api-reference/agent-endpoint/update-an-agent-endpoint /api-reference/openapi.documented.json put /v1/agent/endpoint/{endpointId} Updates an existing agent endpoint by its ID. Only environment and outboundDialType can be modified. # Create a new agent Source: https://docs.roark.ai/api-reference/agent/create-a-new-agent /api-reference/openapi.documented.json post /v1/agent Creates a new agent for the authenticated project. # Delete an agent Source: https://docs.roark.ai/api-reference/agent/delete-an-agent /api-reference/openapi.documented.json delete /v1/agent/{agentId} Soft-deletes an agent by its ID. The agent is hidden from all reads and stops being attributed to new calls, but its record and history are retained. Fails with 409 if the agent is still referenced by simulation run plans: cancel those run plans first. Note: if the agent syncs from a provider integration, also exclude it in the integration settings (or delete the integration) so its calls stop reaching Roark. # Get agent by ID Source: https://docs.roark.ai/api-reference/agent/get-agent-by-id /api-reference/openapi.documented.json get /v1/agent/{agentId} Returns a specific agent by its ID. # List agents Source: https://docs.roark.ai/api-reference/agent/list-agents /api-reference/openapi.documented.json get /v1/agent Returns a paginated list of agents for the authenticated project. # Update an agent Source: https://docs.roark.ai/api-reference/agent/update-an-agent /api-reference/openapi.documented.json put /v1/agent/{agentId} Updates an existing agent by its ID. # Authorization Source: https://docs.roark.ai/api-reference/authorization How to authenticate API requests ### Overview To securely access Roark's API endpoints, you need to authenticate your requests using your unique API key. This ensures that only authorized users can interact with our services. Never expose your API key in client-side code, public repositories, or share it with unauthorized users. If you believe your key has been compromised, regenerate it immediately in the [dashboard](/documentation/getting-started/api-keys). ### Authentication Header ```bash Headers theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} Authorization: Bearer YOUR_API_KEY ``` ```json Response theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} { "authenticated": true, "account": "your-account-name" } ``` ### Example Requests ```bash cURL theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl -X GET "https://api.roark.ai/health" \ -H "Authorization: Bearer YOUR_API_KEY" ``` ```python Python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import requests headers = { "Authorization": "Bearer YOUR_API_KEY" } response = requests.get("https://api.roark.ai/health", headers=headers) ``` ```typescript TypeScript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} const response = await fetch("https://api.roark.ai/health", { headers: { "Authorization": `Bearer YOUR_API_KEY` } }); ``` ### Security Best Practices Store API keys securely in environment variables or a secure key management system Periodically rotate your API keys to maintain security ### Need an API Key? Follow our guide to generate your API key and start making authenticated requests # Append tool invocations to a call Source: https://docs.roark.ai/api-reference/call/append-tool-invocations-to-a-call /api-reference/openapi.documented.json post /v1/call/{callId}/tool-invocations Attach tool invocations that fired during a call to an already-existing call, asynchronously after the call was created. Use this when the tool-call data becomes available later than the call itself (e.g. a Roark simulation, or a call submitted via POST /v1/call before its tools were ready). Writes are idempotent: re-sending an invocation already present on the call (same tool name and timing) is skipped, so retries converge instead of double-counting. Optionally pass `metrics` to (re)score metrics over the newly attached tools, which triggers a billed metric collection job and requires the `metric:create` permission. # Create a call Source: https://docs.roark.ai/api-reference/call/create-a-call /api-reference/openapi.documented.json post /v1/call Create a new call with recording, transcript, agents, and customers # Get a call by ID Source: https://docs.roark.ai/api-reference/call/get-a-call-by-id /api-reference/openapi.documented.json get /v1/call/{callId} Retrieve an existing call by its unique identifier # Get call transcript Source: https://docs.roark.ai/api-reference/call/get-call-transcript /api-reference/openapi.documented.json get /v1/call/{callId}/transcript Fetch the full transcript for a specific call. Optionally specify a transcription source; otherwise the best available source is used automatically. # List call metrics Source: https://docs.roark.ai/api-reference/call/list-call-metrics /api-reference/openapi.documented.json get /v1/call/{callId}/metrics Fetch all call-level metrics for a specific call, including both system-generated and custom metrics. Only returns rows from the **latest** metric-collection job per metric — if the same metric has been recomputed, prior runs are excluded and remain in the metric history. By default returns only successfully computed metrics; pass `?status=all` to also include rows that resolved as NOT_APPLICABLE / DATA_MISSING / ERROR (the `value` field is omitted on those entries — check `captureStatus`). # List call sentiment runs Source: https://docs.roark.ai/api-reference/call/list-call-sentiment-runs /api-reference/openapi.documented.json get /v1/call/{callId}/sentiment-run Fetch detailed sentiment analysis results for a specific call, including emotional tone, key phrases, and sentiment scores. # List calls Source: https://docs.roark.ai/api-reference/call/list-calls /api-reference/openapi.documented.json get /v1/call Returns a paginated list of calls for the authenticated project. # Create a chat Source: https://docs.roark.ai/api-reference/chat/create-a-chat /api-reference/openapi.documented.json post /v1/chat Create a new chat with segments (messages and tool invocations) # Get a chat by ID Source: https://docs.roark.ai/api-reference/chat/get-a-chat-by-id /api-reference/openapi.documented.json get /v1/chat/{id} Retrieve an existing chat by its unique identifier # Get chat transcript Source: https://docs.roark.ai/api-reference/chat/get-chat-transcript /api-reference/openapi.documented.json get /v1/chat/{id}/transcript Fetch the full transcript (messages) for a specific chat. # List chat metrics Source: https://docs.roark.ai/api-reference/chat/list-chat-metrics /api-reference/openapi.documented.json get /v1/chat/{id}/metrics Fetch all metrics for a specific chat, including both system-generated and custom metrics. Only returns rows from the **latest** metric-collection job per metric — if the same metric has been recomputed, prior runs are excluded and remain in the metric history. By default returns only successfully computed metrics; pass `?status=all` to also include rows that resolved as NOT_APPLICABLE / DATA_MISSING / ERROR (the `value` field is omitted on those entries — check `captureStatus`). # List chats Source: https://docs.roark.ai/api-reference/chat/list-chats /api-reference/openapi.documented.json get /v1/chat Returns a paginated list of chats for the authenticated project. # Exchange a CLI authorization code for an API key Source: https://docs.roark.ai/api-reference/cli-auth/exchange-a-cli-authorization-code-for-an-api-key /api-reference/openapi.documented.json post /v1/cli/auth/token Completes the CLI browser-login flow: verifies the single-use authorization code and its PKCE code_verifier, then mints and returns the approved API key exactly once. # Apply a config bundle Source: https://docs.roark.ai/api-reference/config/apply-a-config-bundle /api-reference/openapi.documented.json post /v1/config/apply Reconcile a config-as-code bundle into the project. Submit the full desired set of resources; resources already managed by config are updated, new ones created, and (unless prune is false) config-managed resources absent from the bundle are deleted. Identity is by name — no ids in the bundle. # Diff a config bundle Source: https://docs.roark.ai/api-reference/config/diff-a-config-bundle /api-reference/openapi.documented.json post /v1/config/diff Dry run for a config-as-code apply: returns the projected changes (create / update / delete) for the submitted bundle without writing anything. Submit the full desired set of resources; identity is by name — no ids in the bundle. Run this before apply to preview what would change. # Add an edge case Source: https://docs.roark.ai/api-reference/customer-flow-edge-case/add-an-edge-case /api-reference/openapi.documented.json post /v1/customer-flow/{flowId}/edge-case Adds a variant to an IMPROV flow. A scripted flow's variants are owned by the path engine, one per path through the graph, so they are created by editing the graph through PUT /v1/customer-flow/{flowId}/graph rather than here. Leave personaOverrideId or environmentId unset to inherit the happy path's. # Promote an edge case to the happy path Source: https://docs.roark.ai/api-reference/customer-flow-edge-case/promote-an-edge-case-to-the-happy-path /api-reference/openapi.documented.json post /v1/customer-flow/{flowId}/edge-case/{edgeCaseId}/promote Makes this edge case the flow's happy path, and the outgoing happy path an edge case. Its persona and environment are baked into it first, so edge cases that were inheriting keep the configuration they had. # Remove an edge case Source: https://docs.roark.ai/api-reference/customer-flow-edge-case/remove-an-edge-case /api-reference/openapi.documented.json delete /v1/customer-flow/{flowId}/edge-case/{edgeCaseId} Soft-deletes a variant. On a scripted flow the path engine re-creates a variant for any path still in the graph, so remove the path through PUT /graph instead if that is what you meant. # Update an edge case Source: https://docs.roark.ai/api-reference/customer-flow-edge-case/update-an-edge-case /api-reference/openapi.documented.json put /v1/customer-flow/{flowId}/edge-case/{edgeCaseId} Updates an edge case's title, persona, environment, brief, preceded-by link or expectations. Omitted fields are left alone; `additionalExpectations` replaces the set wholesale rather than appending. Promoting it to the happy path is a separate call, since that also demotes the incumbent. # Create a customer flow Source: https://docs.roark.ai/api-reference/customer-flow/create-a-customer-flow /api-reference/openapi.documented.json post /v1/customer-flow Creates a customer flow. A SCRIPTED flow carries a step graph and gets one way of running it per path through the graph; an IMPROV flow carries the briefs you send. Customer flows replace the older simulation scenarios, so build a flow for anything new. # Delete a customer flow Source: https://docs.roark.ai/api-reference/customer-flow/delete-a-customer-flow /api-reference/openapi.documented.json delete /v1/customer-flow/{flowId} Soft-deletes a customer flow along with its edge cases, expectations and (for scripted flows) its step graph. Run plans that linked it drop it from their test cases. # Get customer flow by ID Source: https://docs.roark.ai/api-reference/customer-flow/get-customer-flow-by-id /api-reference/openapi.documented.json get /v1/customer-flow/{flowId} Returns a customer flow with its happy path, edge cases, expectations and linked agents. Scripted flows also carry their step graph. # List customer flows Source: https://docs.roark.ai/api-reference/customer-flow/list-customer-flows /api-reference/openapi.documented.json get /v1/customer-flow Returns a paginated list of customer flows with their agents, expectations, happy path and edge cases. The step graph is the one field omitted: reading it walks the project's whole step graph, so it comes back from the single-flow endpoint instead. Customer flows are how a project describes what to test; they replace the older simulation scenarios. # Replace a scripted flow's steps Source: https://docs.roark.ai/api-reference/customer-flow/replace-a-scripted-flows-steps /api-reference/openapi.documented.json put /v1/customer-flow/{flowId}/graph Replaces a scripted flow's conversation graph with the tree you send. This is a full replace, not a merge: a step you omit is removed. Include `nodeId` on a step to update the existing one, omit it to create a new step. Where two branches rejoin, keep the `mergeIntoNodeIds` references a read gave you. Dropping them un-merges those branches and is refused unless `allowUnmerge` is set. A change to the set of paths re-seeds how the flow runs, which the response reports as `variantsReshaped` along with the resulting happy path and edge cases. # Update a customer flow Source: https://docs.roark.ai/api-reference/customer-flow/update-a-customer-flow /api-reference/openapi.documented.json put /v1/customer-flow/{flowId} Updates a flow's title, description, branching mode, linked agents or flow-level expectations. The step graph is replaced through PUT /graph. # Update a flow's happy path Source: https://docs.roark.ai/api-reference/customer-flow/update-a-flows-happy-path /api-reference/openapi.documented.json put /v1/customer-flow/{flowId}/happy-path Updates the happy path's title, persona, environment, brief or expectations. Omitted fields are left alone; `additionalExpectations` replaces the set wholesale rather than appending. Its persona and environment are what the edge cases inherit, so changing them here changes every edge case that does not name its own. # Get API health status Source: https://docs.roark.ai/api-reference/health/get-api-health-status /api-reference/openapi.documented.json get /health Returns the health status of the API and its dependencies # Create HTTP request definition Source: https://docs.roark.ai/api-reference/http-request-definition/create-http-request-definition /api-reference/openapi.documented.json post /v1/http-request-definition Creates a new HTTP request definition. The signing secret is only returned in this response and cannot be retrieved later. # Get HTTP request definition by ID Source: https://docs.roark.ai/api-reference/http-request-definition/get-http-request-definition-by-id /api-reference/openapi.documented.json get /v1/http-request-definition/{definitionId} Returns a specific HTTP request definition by its ID. # List HTTP request definitions Source: https://docs.roark.ai/api-reference/http-request-definition/list-http-request-definitions /api-reference/openapi.documented.json get /v1/http-request-definition Returns a paginated list of HTTP request definitions for the authenticated project. # Update HTTP request definition Source: https://docs.roark.ai/api-reference/http-request-definition/update-http-request-definition /api-reference/openapi.documented.json put /v1/http-request-definition/{definitionId} Updates an existing HTTP request definition. # Introduction Source: https://docs.roark.ai/api-reference/introduction Integrating with the Roark API ## Welcome to the Roark API Our API provides programmatic access to all Roark features — from call monitoring and metrics collection to simulation testing. Haven't generated an API key yet? [Generate one here](/documentation/getting-started/api-keys) ## Core Endpoints Upload and analyze call recordings Define metrics and collect results on calls Test agents with synthetic callers ## Getting Started Create your API key in the [dashboard](/documentation/getting-started/api-keys) to start making authenticated requests. Explore our available endpoints and their functionalities in the sections above. Test the API with a simple health check: ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl -X GET https://api.roark.ai/v1/health \ -H "Authorization: Bearer YOUR_API_KEY" ``` ## SDKs and Libraries Official Node.js SDK for Roark Python integration library ## Request constraints Each operation documents its own request constraints and pagination parameters in this reference. The OpenAPI schema beside a field is the source of truth for its length, numeric, and array bounds, including whether a list endpoint uses a cursor or an offset. ## OpenAPI Specification The full OpenAPI 3.1 document is served unauthenticated, and every limit the API enforces appears in it as `maxLength`, `maximum` or `maxItems` on the field itself. It is available at three equivalent URLs: ``` https://api.roark.ai/openapi.json https://api.roark.ai/.well-known/openapi.json https://api.roark.ai/doc ``` Explore our complete OpenAPI specification for detailed endpoint documentation # Create issue Source: https://docs.roark.ai/api-reference/issue/create-issue /api-reference/openapi.documented.json post /v1/issue Opens a new issue with `source: API`. The supplied evidence (calls / chats) is attached in the same transaction; an invalid reference fails the whole request rather than leaving a half-created issue. # Get issue by ID Source: https://docs.roark.ai/api-reference/issue/get-issue-by-id /api-reference/openapi.documented.json get /v1/issue/{issueId} Returns a single issue with its evidence rows. # List issues Source: https://docs.roark.ai/api-reference/issue/list-issues /api-reference/openapi.documented.json get /v1/issue Returns the project’s issues, ordered newest-first. Supports filtering by status, source, and severity, with offset pagination. # Create a knowledge base Source: https://docs.roark.ai/api-reference/knowledge-base/create-a-knowledge-base /api-reference/openapi.documented.json post /v1/knowledge-base Creates a knowledge base. TEXT and JSON sources accept inline `content`. FILE sources require `filename`, `mimeType`, and base64-encoded `contentBase64` — the file is decoded server-side, stored in S3, and (for PDFs) text-extracted inline before the response returns. # Get a knowledge base by ID Source: https://docs.roark.ai/api-reference/knowledge-base/get-a-knowledge-base-by-id /api-reference/openapi.documented.json get /v1/knowledge-base/{id} Returns metadata for a single knowledge base. Does not include content — content is read by metrics at evaluation time and is not exposed over the public API. # List knowledge bases Source: https://docs.roark.ai/api-reference/knowledge-base/list-knowledge-bases /api-reference/openapi.documented.json get /v1/knowledge-base Returns a cursor-paginated list of knowledge bases for the authenticated project. Soft-deleted knowledge bases are excluded. # Create and run a metric collection job Source: https://docs.roark.ai/api-reference/metric-collection-job/create-and-run-a-metric-collection-job /api-reference/openapi.documented.json post /v1/metric/collection-jobs Creates a metric collection job for the specified calls or chats and metrics, then triggers processing. Provide exactly one of callIds or chatIds. # Get metric collection job by ID Source: https://docs.roark.ai/api-reference/metric-collection-job/get-metric-collection-job-by-id /api-reference/openapi.documented.json get /v1/metric/collection-jobs/{jobId} Returns a specific metric collection job with progress information. # Get metric values produced by a metric collection job Source: https://docs.roark.ai/api-reference/metric-collection-job/get-metric-values-produced-by-a-metric-collection-job /api-reference/openapi.documented.json get /v1/metric/collection-jobs/{jobId}/metric-values Returns the metric values produced by the specified job, grouped by metric definition. Unlike `GET /v1/call/:callId/metrics` (which returns the latest values on a call, regardless of which job computed them), this endpoint returns exactly the values *this* job produced — including for calls whose live values have since been overwritten by a later job. By default returns only SUCCESS rows; pass `?status=all` to also include NOT_APPLICABLE / DATA_MISSING / ERROR. # List metric collection jobs Source: https://docs.roark.ai/api-reference/metric-collection-job/list-metric-collection-jobs /api-reference/openapi.documented.json get /v1/metric/collection-jobs Returns a paginated list of metric collection jobs for the project. # Retry a metric collection job Source: https://docs.roark.ai/api-reference/metric-collection-job/retry-a-metric-collection-job /api-reference/openapi.documented.json post /v1/metric/collection-jobs/{jobId}/retry Creates a new metric collection job using the same conversations and metrics as a previous job, then triggers processing. The previous job must be in a terminal state (COMPLETED, FAILED, or CANCELED). Returns the newly created job — track its id for downstream fetches. # Create a metric policy Source: https://docs.roark.ai/api-reference/metric-policy/create-a-metric-policy /api-reference/openapi.documented.json post /v1/metric/policies Creates a new metric policy. Policies define which metrics to collect and under what conditions. # Delete a metric policy Source: https://docs.roark.ai/api-reference/metric-policy/delete-a-metric-policy /api-reference/openapi.documented.json delete /v1/metric/policies/{policyId} Soft-deletes a metric policy. System policies cannot be deleted. # Get metric policy by ID Source: https://docs.roark.ai/api-reference/metric-policy/get-metric-policy-by-id /api-reference/openapi.documented.json get /v1/metric/policies/{policyId} Returns a specific metric policy with its conditions and metrics. # List metric policies Source: https://docs.roark.ai/api-reference/metric-policy/list-metric-policies /api-reference/openapi.documented.json get /v1/metric/policies Returns a paginated list of metric policies for the project, including system policies. # Update a metric policy Source: https://docs.roark.ai/api-reference/metric-policy/update-a-metric-policy /api-reference/openapi.documented.json put /v1/metric/policies/{policyId} Updates an existing metric policy. System policies cannot be modified. # Create a metric variant Source: https://docs.roark.ai/api-reference/metric/create-a-metric-variant /api-reference/openapi.documented.json post /v1/metric/definitions/{idOrSlug}/variants Add a configuration of this metric for your organization, seeded from its Default. Edit it with PUT to change what it measures, then pin it where you want it used. Threshold metrics have no variants: their configuration comes from the metric they derive from. Metrics in a package that manages its own variants reject this too. # Create a threshold on a metric Source: https://docs.roark.ai/api-reference/metric/create-a-threshold-on-a-metric /api-reference/openapi.documented.json post /v1/metric/definitions/{idOrSlug}/thresholds Create a boolean threshold derived from an existing metric. The source metric is addressed by its UUID or its stable `slug`. The threshold fires when the source metric meets the comparison condition. Scope and supported contexts are inherited from the source metric. # Create custom metric definition Source: https://docs.roark.ai/api-reference/metric/create-custom-metric-definition /api-reference/openapi.documented.json post /v1/metric/definitions Create a new metric definition. The `calculationType` field selects the variant: LLM_JUDGE (LLM-evaluated), FORMULA (computed from a math expression over other metrics), or PATTERN (detects a trigger→outcome pattern within a window). To create a threshold on top of an existing metric, use `POST /metric/definitions/{idOrSlug}/thresholds` instead. # Delete a metric definition Source: https://docs.roark.ai/api-reference/metric/delete-a-metric-definition /api-reference/openapi.documented.json delete /v1/metric/definitions/{idOrSlug} Archives (soft-deletes) a custom metric definition, addressed by its UUID or its stable `slug`. The metric is hidden from all reads and stops being collected, but its record and previously collected values are retained. System metrics cannot be deleted. Fails with 409 if the metric is still used as a source by a derived metric (delete those first). # Delete a metric variant Source: https://docs.roark.ai/api-reference/metric/delete-a-metric-variant /api-reference/openapi.documented.json delete /v1/metric/definitions/{idOrSlug}/variants/{variantId} Remove one of your organization’s variants. Anything pinned to it falls back to the Default, so deleting a fork of a Roark variant returns you to Roark’s configuration. Roark’s own variants cannot be deleted, and neither can a Default. Values already collected under the deleted variant are retained. # Get a metric definition Source: https://docs.roark.ai/api-reference/metric/get-a-metric-definition /api-reference/openapi.documented.json get /v1/metric/definitions/{idOrSlug} Fetch a single metric definition by its UUID or its stable `slug` (e.g. `customer_satisfaction`). Resolution is scoped to the project — a project-owned metric wins over an org-wide one, which wins over a system metric of the same `slug`. # Get a metric variant Source: https://docs.roark.ai/api-reference/metric/get-a-metric-variant /api-reference/openapi.documented.json get /v1/metric/definitions/{idOrSlug}/variants/{variantId} One configuration of this metric, by id. # List a metric’s variants Source: https://docs.roark.ai/api-reference/metric/list-a-metric’s-variants /api-reference/openapi.documented.json get /v1/metric/definitions/{idOrSlug}/variants Every configuration of this metric your organization can use: Roark’s own variants and any your organization has added or forked. `isDefault` marks the one the metric is scored with when nothing pins another; pass any variant’s `id` as `sourceVariantId` to pin it on a derived metric. Auto-managed variants (the ones a package materializes for you) are not listed: they are engine state, not configuration you author. # List metric definitions Source: https://docs.roark.ai/api-reference/metric/list-metric-definitions /api-reference/openapi.documented.json get /v1/metric/definitions Fetch metric definitions available in the project, including both system-generated and custom metrics. Results are ordered by immutable definition ID; pass `nextCursor` to retrieve the following page. # Update a metric definition Source: https://docs.roark.ai/api-reference/metric/update-a-metric-definition /api-reference/openapi.documented.json put /v1/metric/definitions/{idOrSlug} Update the editable subset of a custom metric definition, addressed by its UUID or its stable `slug`. Only the supplied fields are changed; omitted fields are left unchanged. Every update creates a new immutable version; the response carries the advanced `versionId`. Immutable fields (scope, outputType, calcType, …) are rejected, and which fields are editable depends on the metric (e.g. derived metrics only allow `name`). Roark's own metrics are rejected here: this endpoint edits the shared definition, which every workspace sees. To change one for your workspace alone, edit its variant with PUT /v1/metric/definitions/{idOrSlug}/variants/{variantId}, which forks it for you and leaves every other workspace on the original. # Update a metric variant Source: https://docs.roark.ai/api-reference/metric/update-a-metric-variant /api-reference/openapi.documented.json put /v1/metric/definitions/{idOrSlug}/variants/{variantId} Rename a variant, change its configuration, or both. Every configuration change creates a new immutable version and advances `versionId`; the response carries the advanced value. **Editing one of Roark’s own variants forks it for your organization.** The response carries the new variant’s `id`, which will differ from the one in the path, and `isSystem` becomes false. Roark’s variant is untouched and other organizations keep it. DELETE your fork to go back to it. Which fields are editable depends on the metric: some Roark metrics lock their prompt or output configuration, and a locked field is rejected rather than ignored. # Get environment by ID Source: https://docs.roark.ai/api-reference/simulation-environment/get-environment-by-id /api-reference/openapi.documented.json get /v1/simulation/environment/{environmentId} Returns a single environment by its ID. # List environments Source: https://docs.roark.ai/api-reference/simulation-environment/list-environments /api-reference/openapi.documented.json get /v1/simulation/environment Returns a paginated list of environments: the project's own plus the environments Roark curates and shares across every project. Reference one by id when setting a customer flow variant's environment. # Get simulation by ID Source: https://docs.roark.ai/api-reference/simulation-job/get-simulation-by-id /api-reference/openapi.documented.json get /v1/simulation/job/{jobId} Get a individual simulation run directly by its ID. This is generally part of a larger simulation run plan job. # Lookup by phone number Source: https://docs.roark.ai/api-reference/simulation-job/lookup-by-phone-number /api-reference/openapi.documented.json get /v1/simulation/job/lookup Find the matching simulation using the number used by the Roark simulation agent. # Create a new persona Source: https://docs.roark.ai/api-reference/simulation-persona/create-a-new-persona /api-reference/openapi.documented.json post /v1/persona Creates a new persona for the authenticated project. # Delete a persona Source: https://docs.roark.ai/api-reference/simulation-persona/delete-a-persona /api-reference/openapi.documented.json delete /v1/persona/{personaId} Soft-deletes a persona by its ID. The persona is hidden from all reads and stops being available to new simulations, but its record and history are retained. System personas cannot be deleted. # Get persona by ID Source: https://docs.roark.ai/api-reference/simulation-persona/get-persona-by-id /api-reference/openapi.documented.json get /v1/persona/{personaId} Returns a specific persona by its ID. # List personas Source: https://docs.roark.ai/api-reference/simulation-persona/list-personas /api-reference/openapi.documented.json get /v1/persona Returns a paginated list of personas for the authenticated project. # Update a persona Source: https://docs.roark.ai/api-reference/simulation-persona/update-a-persona /api-reference/openapi.documented.json put /v1/persona/{personaId} Updates an existing persona by its ID. # Get simulation plan job Source: https://docs.roark.ai/api-reference/simulation-run-plan-job/get-simulation-plan-job /api-reference/openapi.documented.json get /v1/simulation/plan/job/{jobId} Retrieve details of a simulation plan job including all associated simulation jobs (calls) # List simulation plan jobs Source: https://docs.roark.ai/api-reference/simulation-run-plan-job/list-simulation-plan-jobs /api-reference/openapi.documented.json get /v1/simulation/plan/jobs Returns a paginated list of simulation run plan jobs. Filter by status, plan ID, or label to find specific simulation batches. # Run a simulation plan Source: https://docs.roark.ai/api-reference/simulation-run-plan-job/run-a-simulation-plan /api-reference/openapi.documented.json post /v1/simulation/plan/{planId}/job Deprecated: use POST /v1/simulation/run, which does the same thing and can also take the plan configuration inline, so a one-off run does not have to create a plan first. Creates and executes a job for an existing simulation run plan. Optionally provide runtime variables to override plan-defined variables. # Create a run plan Source: https://docs.roark.ai/api-reference/simulation-run-plan/create-a-run-plan /api-reference/openapi.documented.json post /v1/simulation/plan Creates a new simulation run plan. To run a simulation, use POST /v1/simulation/run instead: it starts a run from a plan or from an inline configuration, and takes runtime variables. Create a plan here when you want a reusable, named one to run later. # Delete a run plan Source: https://docs.roark.ai/api-reference/simulation-run-plan/delete-a-run-plan /api-reference/openapi.documented.json delete /v1/simulation/plan/{planId} Soft-deletes a simulation run plan by its ID. # Get run plan by ID Source: https://docs.roark.ai/api-reference/simulation-run-plan/get-run-plan-by-id /api-reference/openapi.documented.json get /v1/simulation/plan/{planId} Returns a specific simulation run plan by its ID. # List run plans Source: https://docs.roark.ai/api-reference/simulation-run-plan/list-run-plans /api-reference/openapi.documented.json get /v1/simulation/plan Returns a paginated list of simulation run plans. Optionally filter by search text or agent ID. # Update a run plan Source: https://docs.roark.ai/api-reference/simulation-run-plan/update-a-run-plan /api-reference/openapi.documented.json put /v1/simulation/plan/{planId} Updates an existing simulation run plan by its ID. # Run a simulation Source: https://docs.roark.ai/api-reference/simulation/run-a-simulation /api-reference/openapi.documented.json post /v1/simulation/run Starts a simulation and returns the run. Send `plan` to describe a simulation and run it once. Add `saveAsPlan` to keep that configuration as a reusable run plan. Send `planId` instead to run a plan you already have. # Create webhook Source: https://docs.roark.ai/api-reference/webhook/create-webhook /api-reference/openapi.documented.json post /v1/webhook Creates a new webhook with event subscriptions. The signing secret is only returned in this response. # Delete webhook Source: https://docs.roark.ai/api-reference/webhook/delete-webhook /api-reference/openapi.documented.json delete /v1/webhook/{webhookId} Deletes a webhook and all its event subscriptions. # Event Source: https://docs.roark.ai/api-reference/webhook/event /api-reference/openapi.documented.json webhook Webhook Roark POSTs a JSON payload to every endpoint subscribed to one of the events listed below. Every payload uses the same envelope (`event`, `version`, `timestamp`, `data`); the `event` field is the discriminator that selects the matching `data` shape. Acknowledge with any 2xx response within 10 seconds. Non-2xx responses and timeouts are retried with exponential backoff. **Events** - `call.analysis.completed` - `call.analysis.failed` - `call.analysis.cancelled` - `call.evaluation.completed` - `call.evaluation.failed` - `simulation.run_plan_job.started` - `simulation.run_plan_job.completed` - `simulation.run_plan_job.failed` - `simulation.run_plan_job.cancelled` - `simulation.job.started` - `simulation.job.completed` - `simulation.job.failed` - `simulation.job.cancelled` - `metric_collection.job.completed` - `metric_collection.job.failed` - `chat.analysis.completed` - `chat.analysis.failed` - `issue.opened` - `issue.resolved` # Get webhook by ID Source: https://docs.roark.ai/api-reference/webhook/get-webhook-by-id /api-reference/openapi.documented.json get /v1/webhook/{webhookId} Returns a specific webhook with its event subscriptions. # List webhooks Source: https://docs.roark.ai/api-reference/webhook/list-webhooks /api-reference/openapi.documented.json get /v1/webhook Returns a paginated list of webhooks with their event subscriptions. # Agents Source: https://docs.roark.ai/documentation/config-as-code/agents Define voice agents as config A voice agent. Thin by design; endpoints are nested and fanned out on apply. References nothing. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: agent name: frontdesk description: Bright Smiles front desk booking agent customId: bright-smiles-frontdesk endpoints: - name: outbound-primary value: '+15551234567' direction: INCOMING_AND_OUTGOING ``` * `name` is the local key (`^[a-z0-9][a-z0-9-_.]*$`) and, with the kind, forms the config identity. * `endpoints[]` each take a `name`, a `value` (E.164 phone number), a `direction` (`INCOMING`, `OUTGOING`, or `INCOMING_AND_OUTGOING`), and an optional `environment`. * Other resources reference an agent by its `name` (e.g. a flow's `agents:` list, or a collector's `AGENT` filter). For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Alerts Source: https://docs.roark.ai/documentation/config-as-code/alerts Define alerts (monitors) as config: threshold, event, and simulation triggers An alert (monitor) reacts to a trigger by opening an issue and/or notifying Slack channels and webhooks. Pick one of three trigger types with `trigger.type`; the type is immutable once created (a change is delete-and-recreate). ## Threshold Watch a metric aggregate over a rolling window. Opens an issue when it crosses the threshold, and optionally notifies Slack/webhooks. `metric` is the metric's slug (a custom metric you defined, or a Roark system metric); the variant defaults to the metric's default. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: alert name: high-frustration trigger: type: threshold metric: frustration_score # metric slug (its metricId) aggregation: MEAN # COUNT | RATE_PER_MINUTE | MEAN windowMinutes: 60 operator: GT # GT | GTE | LT | LTE thresholdValue: 3 minSampleSize: 5 # optional; skip windows with fewer data points actions: slack: - channelId: C0123ABCXYZ # Slack-native channel id channelName: '#agent-quality' webhooks: - https://hooks.example.com/roark/alerts ``` ## Event Fire on platform events (call/chat analysis, simulation jobs, metric collection, issues). ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: alert name: analysis-failures trigger: type: event events: - CALL_ANALYSIS_FAILED - CHAT_ANALYSIS_FAILED actions: slack: - channelId: C0123ABCXYZ channelName: '#platform-alerts' ``` ## Simulation Notify on a simulation run-plan job outcome. `runPlan` (by name) scopes it to one run plan; omit for project-wide. A simulation alert requires exactly one Slack channel and does not support webhooks. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: alert name: nightly-suite-result trigger: type: simulation conditions: [FAILURE, THRESHOLD_FAILED] # SUCCESS | FAILURE | THRESHOLD_FAILED runPlan: nightly-regression deliveryFormat: MESSAGE # MESSAGE | PDF actions: slack: - channelId: C0999SIMS channelName: '#sim-results' ``` * **Threshold alerts always open an issue**; Slack and webhook delivery are optional additions. * **Slack channels** are referenced by their Slack-native `channelId` (e.g. `C0123ABC`) plus a `channelName`. The Slack workspace is resolved from the project's connected Slack integration, so no UUIDs go in config. * **Webhooks** must be public `http(s)` URLs. Referencing a Slack channel requires a **Slack integration connected to the project** in Roark first. Connect Slack under the project's integration settings, then reference the channel by id here. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Collectors Source: https://docs.roark.ai/documentation/config-as-code/collectors Decide which metrics get collected on which conversations Decides which metrics get collected on which conversations (the config form of a [collector](/documentation/metrics/metric-collectors)). Its `metrics:` list references both custom metrics you defined and Roark's system metrics, all by slug. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: collector name: consent-on-frontdesk modality: call status: ACTIVE metrics: - consent_collection_consent_obtained - refund_policy_accuracy filters: - conditions: - type: AGENT key: frontdesk operator: EQUALS ``` * `modality` is `call` or `chat` and is immutable once created (make a new collector to switch). * **Metrics** are referenced by their stable slug, not a UUID, and must be visible to the project and support the collector's `modality`. * **Filters** are condition groups: groups OR together, conditions within a group AND. An `AGENT` condition's `key` is a config-managed agent name (resolved on apply); every other type (`CALL_SOURCE`, `CALL_PROPERTY`, `INTEGRATION`) uses its `key`/`value` verbatim. Omit `filters` to score every matching conversation. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Flows Source: https://docs.roark.ai/documentation/config-as-code/flows Define simulation flows as config (improv and scripted) A simulation flow. Both types share `kind: flow` and are discriminated by `type`. Flows reference agents, a persona, and an environment by name. ## Improv (`type: improv`) An improvised simulation with a happy path and edge-case variants. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: flow type: improv name: frustrated-rebooking agents: [frontdesk] happyPath: persona: frustrated-caller environment: Quiet line prompt: You call to rebook the cleaning that was cancelled on you. edgeCases: - name: escalates-to-manager prompt: file://prompts/escalates-to-manager.md expectations: - Agent offers to escalate rather than arguing ``` * `happyPath` requires a `persona` and `environment` (by name); `prompt` and `expectations` are optional. * Each `edgeCases[]` entry inherits the happy path unless it overrides `persona` / `environment` / `prompt`, and can add its own `expectations`. ## Scripted (`type: scripted`) A step-by-step conversation graph (branches, merges, DTMF, voicemail, scenario links). Each apply replaces the whole graph. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: flow type: scripted name: booking-scripted agents: [frontdesk] branchingMode: ADAPTIVE graph: - ref: greeting type: AGENT_TURN content: Thanks for calling, how can I help? steps: - ref: request type: CUSTOMER_TURN content: I'd like to book a cleaning. ``` * Each node has a `type` and an optional `ref` label so other nodes can rejoin it via `mergeInto` (DAG merge edges). `steps` are a node's successors (more than one = a branch). * No UUIDs: identity is the `ref`, and each apply replaces the whole graph. * The environment is referenced by its display name and must already exist in Roark. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Metrics Source: https://docs.roark.ai/documentation/config-as-code/metrics Define custom LLM-judged metrics as config A [custom metric](/documentation/metrics/custom-metrics) you author, graded by an LLM against a prompt. Reference it from a collector by its `name`. ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: metric name: refund_policy_accuracy # the slug a collector references displayName: Refund Policy Accuracy type: BOOLEAN # BOOLEAN | SCALE | NUMERIC | TEXT | CLASSIFICATION prompt: | Using {{transcript}} and {{world_context}}, did the agent state the refund policy correctly? trueLabel: Accurate falseLabel: Inaccurate ``` * `name` is the metric's stable slug (its `metricId`), exactly what a collector references. `displayName` is the human name shown in the UI. * The **prompt** is the grading criteria and must reference a transcript source (e.g. `{{transcript}}`) and `{{world_context}}`. Inline or `file://`. * `name`, `type`, and `scope` are immutable once created (a change is create-new). Editing the prompt, labels, bounds, or options creates a new version. * SCALE metrics add `scaleMin` / `scaleMax` (+ optional `scaleLabels`); CLASSIFICATION metrics add `options` (+ optional `maxSelections`); `scope: PER_PARTICIPANT` requires a `participantRole`. **System metrics** (Roark's built-in metrics) are managed by Roark, not by you: you never define them in config. To collect one, just reference its slug in a collector's `metrics:` list, exactly like a custom metric. The same is true for derived metrics (thresholds, formulas, patterns), which are not yet definable in config. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Overview Source: https://docs.roark.ai/documentation/config-as-code/overview Define your Roark agents, personas, flows, metrics, collectors, and alerts as YAML in git and deploy them with one apply Config as Code lets you define your Roark resources - agents, personas, simulation flows, custom metrics, collectors, and alerts - as YAML files in your own git repository, then deploy them with a single apply. Your config repo is the source of truth: Roark reconciles the live project to match what you submitted, creating what's new, updating what changed, and removing what you deleted. ## Quickstart Put each resource in a YAML file under a directory, then apply. Install the [CLI](/documentation/sdks/cli) and set `ROARK_API_BEARER_TOKEN` first (the key needs `config:apply`). ```yaml roark/agents/frontdesk.yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} kind: agent name: frontdesk ``` ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} roark config diff ./roark # preview the changes roark config apply ./roark # apply them ``` That's the whole loop. Add personas, flows, metrics, collectors, and alerts the same way; each kind has its own reference page below. *** Config as Code manages **resource definitions**. It does not run simulations or place calls; you trigger those as usual once the resources exist (see [CI/CD](/documentation/simulation-testing/ci-cd) to apply config and start a run in one pipeline). You write only a human-readable `name` for each resource. Roark derives a stable identity (`configKey = /`) and resolves cross-references by name, so there are no UUIDs in your files and no state file to keep in sync. The easiest way to run Config as Code. `roark config diff ./roark` and `roark config apply ./roark` bundle your directory (resolving `file://` prompts) and submit it for you. See the CLI page for install, auth, and a CI example. *** ## How it works You submit the full desired set of resources to a single endpoint. Roark: 1. **Parses and validates** every resource against the schema. 2. **Diffs** the submitted set against the resources this project already manages via config. 3. **Reconciles**: creates new resources, updates changed ones, and (unless you opt out) deletes config-managed resources you removed from the submission. There are two endpoints: | Endpoint | What it does | | :---------------------- | :---------------------------------------------------------------- | | `POST /v1/config/diff` | **Dry run.** Returns the changes that *would* be made. No writes. | | `POST /v1/config/apply` | **Applies** the changes and returns what happened. | Always run `diff` first to preview the changes, then `apply`. *** ## Prerequisites * A Roark API key with the **`config:apply`** permission. Generate one from [API Keys](/documentation/getting-started/api-keys) and confirm it carries `config:apply`. * A git repository to hold your config files (any layout; Roark reads the files you submit). * The **Roark CLI** (recommended): `npm install -g @roarkanalytics/cli`, or run it on demand with `npx @roarkanalytics/cli`. The CLI builds the bundle from your config directory and drives `diff`/`apply` for you, so it's the easiest way to deploy. Raw HTTP and the SDKs work too. The API key is scoped to a single project. Everything you apply lands in that project. *** ## Repository layout One file per resource, discriminated by `kind`. A conventional layout: ``` roark/ agents/frontdesk.yaml personas/frustrated-caller.yaml flows/frustrated-rebooking.yaml # improv flow flows/booking-scripted.yaml # scripted-graph flow metrics/refund-policy-accuracy.yaml collectors/consent-on-frontdesk.yaml alerts/high-frustration.yaml # threshold / event / simulation alert prompts/escalates-to-manager.md # referenced by file:// ``` Add this header to any resource file for editor autocomplete and validation: ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json ``` *** ## Resource kinds One file per resource, discriminated by `kind`. Each kind has its own reference page with fields and examples: Voice agents and their phone endpoints. The simulated caller for a flow. Simulation flows: improvised or scripted graphs. Custom LLM-judged metric definitions. Which metrics get collected on which conversations. Alerts (monitors): threshold, event, and simulation triggers. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. *** ## Deploying The easiest way to deploy is the **Roark CLI**. Point it at your config directory and it builds the bundle for you (reading every YAML file and inlining `file://` prompts), so there is no JSON body to assemble by hand. The [CLI](/documentation/sdks/cli) does the bundling for you: `roark config diff ./roark` and `roark config apply ./roark` take the directory directly, resolve `file://` prompt references, and submit the result. The raw requests below are what it sends. Give the CLI the project API key that carries `config:apply`: ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} export ROARK_API_BEARER_TOKEN="" # or store it once: roark auth login ``` ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} roark config diff ./roark ``` The CLI reads every resource under `./roark`, builds the bundle, and prints one line per change (`+` create, `~` update, `-` delete) with a tally: ```text theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} + collector/consent-on-frontdesk ~ agent/frontdesk 1 to create, 1 to update, 0 to delete ``` Resources already in sync are no-ops and aren't listed, so a project that fully matches your config prints `0 to create, 0 to update, 0 to delete`. ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} roark config apply ./roark ``` `apply` previews the same diff, asks you to confirm, then reconciles and reports what it did. Pass `--yes` to skip the prompt in CI, and `--no-prune` for an additive-only apply that never deletes. ### Using raw HTTP If you'd rather call the API directly, bundle your resources into a single JSON body: `{ "resources": [...], "prune": true }`, where each entry is one resource in the same shape as its YAML. `POST` it to `/v1/config/diff` first, then `/v1/config/apply`. ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl -X POST https://api.roark.ai/v1/config/diff \ -H "Authorization: Bearer $ROARK_API_KEY" \ -H "Content-Type: application/json" \ --data-binary @bundle.json ``` The response lists each projected change with an `op` (`create`, `update`, or `delete`) plus a summary; in-sync resources are counted in `summary.noop` and omitted from `changes`: ```json theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} { "data": { "changes": [ { "configKey": "collector/consent-on-frontdesk", "kind": "collector", "name": "consent-on-frontdesk", "op": "create" } ], "summary": { "create": 1, "update": 0, "delete": 0, "noop": 0 } } } ``` `apply` takes the same body against `/v1/config/apply`; each change comes back with a `status` (`applied` or `failed`) and, on success, the resource `id`. These endpoints are available in the [Node.js](/documentation/sdks/node-sdk) and [Python](/documentation/sdks/python-sdk) SDKs as `config.diff` and `config.apply`, taking the same bundle, and in the [CLI](/documentation/sdks/cli) as `roark config diff` and `roark config apply`, taking a directory. *** ## Apply semantics * **Identity is by name.** Re-submitting an unchanged resource updates it in place; it never creates a duplicate. Renaming a resource is a delete of the old name plus a create of the new one. * **Cross-references resolve by name** within the same submission (a flow's `agents:`/`persona:`, a collector's `AGENT` filter). The referenced resource must be in the bundle or already config-managed in the project. * **Prune deletes what you removed.** By default, config-managed resources absent from the submission are deleted so the project matches your repo exactly. To layer additive changes without deleting, send `"prune": false`. * **Prompts are code.** Any prompt field takes an inline string or `file://relative/path.md`, resolved relative to your config root and inlined before you submit. * **Idempotent.** Applying the same bundle twice converges to the same state. An unchanged resource is a no-op on the next `diff`/`apply`, not a rewrite, so a re-run of an in-sync project reports no changes. With `prune` enabled (the default), a resource you delete from your repo is deleted from Roark on the next apply. Submit the **full** desired set every time, or use `"prune": false` for additive-only applies. *** ## Config-managed resources in the UI A resource created by config is **read-only in the dashboard** and carries a "managed by config" badge. To change it, edit your config and re-apply. If you need to hand a resource back to manual UI editing, **detach** it (from the resource's menu in the dashboard). Detaching clears its config ownership: * A later apply that still lists it will re-adopt it. * A later apply that omits it will simply leave it alone (it is no longer config-managed, so prune won't touch it). *** ## Recommended workflow 1. Keep your `roark/` config in a git repo, reviewed via pull requests. 2. In CI, run `roark config diff ./roark` on every PR and post the output for review. 3. On merge to your main branch, run `roark config apply ./roark --yes`. Both CI steps read the API key from `ROARK_API_BEARER_TOKEN` (store it as a secret with `config:apply`). This gives you versioned, reviewable, reproducible Roark resources with a full audit trail in git. For a copy-pasteable GitHub Actions workflow that does exactly this, see [Using the CLI in CI](/documentation/sdks/cli#using-the-cli-in-ci). # Personas Source: https://docs.roark.ai/documentation/config-as-code/personas Define the simulated caller as config The simulated caller for a flow. Self-contained (references nothing). ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # yaml-language-server: $schema=https://schema.roark.ai/roark-config.schema.json kind: persona name: frustrated-caller displayName: Dana Whitfield language: EN accent: US gender: FEMALE baseEmotion: FRUSTRATED backstoryPrompt: file://prompts/frustrated-caller.md ``` * `name` is the local key; `displayName` is the name the agent hears (defaults to `name`). * `language`, `accent`, and `gender` are required; behavioral knobs (`baseEmotion`, `speechPace`, `speechClarity`, `intentClarity`, `confirmationStyle`, `memoryReliability`, `responseTiming`, `backgroundNoise`, and the idle-message fields) are optional and default sensibly. * `backstoryPrompt` is the caller's background: inline or a `file://` reference resolved from your config root. * A flow references a persona by its `name`. For the full field reference of every kind, see the [Config DSL reference](https://schema.roark.ai/roark-config.schema.json) schema. # Data Retention Source: https://docs.roark.ai/documentation/enterprise/data-retention Control how long Roark keeps your call data, and whether expired data is hidden or permanently deleted Data retention lets you control how long Roark keeps your calls and their metrics, and what happens when they expire. It's configured **per project**. ## Configure retention Go to **Settings → Project → Data Controls** and set two things: How long to keep calls after they're created. Choose 60, 90, 180, or 365 days, or **Indefinite** to keep data forever. Defaults to **Indefinite**. What happens to a call once it passes the retention period: * **Hide from dashboard (recoverable)** - the call is soft-deleted: hidden from the dashboard but still stored on the backend, so it can be recovered. This is the default. * **Permanently delete (irreversible)** - the call is physically purged everywhere: recordings, transcripts, metrics, and traces. ## How expiry works A call "expires" once its age exceeds the retention period. What happens next depends on the mode: * **Hide** - expired calls are soft-deleted (removed from the dashboard but recoverable). * **Permanent** - expired calls are hidden first, then **physically purged 30 days later** across all systems (Postgres, the metric store, recordings in object storage, analytics, and traces). This grace period gives you a window to catch a misconfiguration before data is gone for good. Permanent deletion cannot be undone. Once a call is physically purged, its recordings, transcripts, metrics, and traces are gone across every system. Retention applies going forward from when you set it. Setting a shorter period will begin expiring older calls that already fall outside the new window. # API Keys Source: https://docs.roark.ai/documentation/getting-started/api-keys Generate and manage authentication keys for Roark APIs ## Overview API keys authenticate your requests to Roark's API, enabling secure access to monitoring, testing, and analytics features. API keys are essential for all integrations - whether you're using our SDKs, webhooks, or direct API calls. ## Generating an API Key Go to the API Keys section in your dashboard Click the "Create API Key" button Choose your access level: * **Read-Only**: Retrieve data only * **Full Access**: Read and write operations Copy and store your key immediately - it won't be shown again Create API Key Modal ## Using Your API Key All API requests require authentication via the Authorization header: ```bash cURL theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl -X GET "https://api.roark.ai/health" \ -H "Authorization: Bearer YOUR_API_KEY" ``` ```python Python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import requests headers = { "Authorization": "Bearer YOUR_API_KEY" } response = requests.get("https://api.roark.ai/health", headers=headers) ``` ```typescript TypeScript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} const response = await fetch("https://api.roark.ai/health", { headers: { "Authorization": `Bearer YOUR_API_KEY` } }); ``` ```javascript Node.js theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} const axios = require('axios'); const config = { headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }; const response = await axios.get('https://api.roark.ai/health', config); ``` ## Security Best Practices Never expose your API key in client-side code, public repositories, or logs. If compromised, regenerate immediately in your dashboard. Store keys in `.env` files or secure vaults, never hardcode them Regenerate keys periodically to maintain security Use read-only keys when write access isn't needed Check last-used timestamps to detect unauthorized access ## Common Integration Patterns Initialize SDKs with your key: ```javascript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} const roark = new RoarkClient({ apiKey: process.env.ROARK_API_KEY }); ``` Store as secret in your pipeline: ```yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} env: ROARK_API_KEY: ${{ secrets.ROARK_API_KEY }} ``` ## Need Help? Explore all available endpoints and parameters Step-by-step platform integration tutorials # Welcome to Roark Source: https://docs.roark.ai/documentation/getting-started/introduction The quality platform for voice and chat AI Roark is where voice and chat agents prove they're ready: **simulate** every failure mode before launch, **analyze** every production call once you're live, and score them all with metrics powered by **Roark Prism**, our purpose-built evaluation model. Break it in staging, not in production. Test agents with synthetic callers across flows, personas, accents, and edge cases before you ship. Every production call scored, filed, and traced. Transcribed and measured against your metrics in real time. Label calls, set ground truth, and align every metric with your team's judgment. From caught issue to drafted fix to proven deploy, with you in the loop. *** ## Your first simulation Pick a path. Each one takes you from nothing to a running simulation, reusing a built-in persona and environment so there's nothing extra to set up. Copy, paste, run with [`@roarkanalytics/sdk`](/documentation/sdks/node-sdk): ```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import Roark from '@roarkanalytics/sdk' const client = new Roark({ bearerToken: process.env.ROARK_API_BEARER_TOKEN }) // An agent and how to reach it const agent = await client.agent.create({ name: 'Front Desk' }) const endpoint = await client.agentEndpoint.create({ agentId: agent.data.id, value: '+15551234567', direction: 'INCOMING_AND_OUTGOING', }) // Reuse a built-in persona and the "Quiet line" environment const personas = await client.simulationPersona.list({ limit: 50 }) const envs = await client.simulationEnvironment.list({ limit: 50 }) const quietLine = envs.data.find((e) => e.name === 'Quiet line')! // The conversation to test const flow = await client.customerFlow.create({ type: 'IMPROV', title: 'Rebooking', agentIds: [agent.data.id], happyPath: { title: 'Frustrated rebooking', personaOverrideId: personas.data[0].id, environmentId: quietLine.id, prompt: 'You call to rebook a cleaning that was cancelled on you.', }, agentExpectations: [{ prompt: 'Agent offers a concrete alternative appointment time' }], }) // Run it const run = await client.simulation.run({ plan: { direction: 'INBOUND', maxSimulationDurationSeconds: 300, agentEndpoints: [{ id: endpoint.data.id }], flows: [{ id: flow.data.id, happyPath: true }], metrics: [{ slug: 'task_completion' }], }, }) console.log(run.data.simulationRunPlanJobId) ``` Watch it in the dashboard, or poll `client.simulationRunPlanJob.getByID(run.data.simulationRunPlanJobId)`. Same flow with [`roark_analytics`](/documentation/sdks/python-sdk): ```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import os from roark_analytics import Roark client = Roark(bearer_token=os.environ["ROARK_API_BEARER_TOKEN"]) # An agent and how to reach it agent = client.agent.create(name="Front Desk") endpoint = client.agent_endpoint.create( agent_id=agent.data.id, value="+15551234567", direction="INCOMING_AND_OUTGOING", ) # Reuse a built-in persona and the "Quiet line" environment personas = client.simulation_persona.list(limit=50) envs = client.simulation_environment.list(limit=50) quiet_line = next(e for e in envs.data if e.name == "Quiet line") # The conversation to test flow = client.customer_flow.create( type="IMPROV", title="Rebooking", agent_ids=[agent.data.id], happy_path={ "title": "Frustrated rebooking", "personaOverrideId": personas.data[0].id, "environmentId": quiet_line.id, "prompt": "You call to rebook a cleaning that was cancelled on you.", }, agent_expectations=[{"prompt": "Agent offers a concrete alternative appointment time"}], ) # Run it run = client.simulation.run( plan={ "direction": "INBOUND", "maxSimulationDurationSeconds": 300, "agentEndpoints": [{"id": endpoint.data.id}], "flows": [{"id": flow.data.id, "happyPath": True}], "metrics": [{"slug": "task_completion"}], }, ) print(run.data.simulation_run_plan_job_id) ``` Drive the same steps from your terminal (install the [CLI](/documentation/sdks/cli) and set `ROARK_API_BEARER_TOKEN`). Commands print JSON, so pipe through `jq`: ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} AGENT_ID=$(roark agent create --name "Front Desk" | jq -r '.data.id') ENDPOINT_ID=$(roark agent endpoint create --agent-id "$AGENT_ID" \ --value "+15551234567" --direction INCOMING_AND_OUTGOING | jq -r '.data.id') PERSONA_ID=$(roark simulation persona list --limit 50 | jq -r '.data[0].id') ENV_ID=$(roark simulation environment list --limit 50 \ | jq -r '.data[] | select(.name=="Quiet line") | .id') FLOW_ID=$(roark customer-flow create --data "$(jq -nc \ --arg a "$AGENT_ID" --arg p "$PERSONA_ID" --arg e "$ENV_ID" '{ type:"IMPROV", title:"Rebooking", agentIds:[$a], happyPath:{ title:"Frustrated rebooking", personaOverrideId:$p, environmentId:$e, prompt:"You call to rebook a cleaning that was cancelled on you." } }')" | jq -r '.data.id') roark simulation run --data "$(jq -nc --arg ep "$ENDPOINT_ID" --arg f "$FLOW_ID" '{ plan:{ direction:"INBOUND", maxSimulationDurationSeconds:300, agentEndpoints:[{id:$ep}], flows:[{id:$f, happyPath:true}], metrics:[{slug:"task_completion"}] } }')" ``` Keep your test suite in git as YAML and apply it with the [CLI](/documentation/sdks/cli): ```yaml roark/flows/rebooking.yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} kind: flow type: improv name: rebooking agents: [frontdesk] happyPath: persona: frustrated-caller environment: Quiet line prompt: You call to rebook a cleaning that was cancelled on you. expectations: - Agent offers a concrete alternative appointment time ``` ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} roark config apply ./roark ``` Config defines your resources; launch a run from the CLI, an SDK, or the dashboard. Full example (agents + personas + flows) and how to run it in CI: **[Config as Code](/documentation/config-as-code/overview)** and **[CI/CD](/documentation/simulation-testing/ci-cd)**. No code: 1. **Connect your agent** under [Agents](/documentation/integrations/overview) (Vapi, Retell, ElevenLabs, LiveKit, and more). 2. **Create a [customer flow](/documentation/simulation-testing/customer-flows)** for the conversation to test (pick a built-in [persona](/documentation/simulation-testing/personas)). 3. **New Run**: pick the agent, attach the flow, add pass/fail [checks](/documentation/metrics/thresholds), launch. 4. **Read the verdict** and per-conversation scores. *** ## Explore the docs Customer flows, personas, templates, plans, and schedules Define agents, personas, flows, metrics, collectors, and alerts as YAML in git Call history, traces, reports, and dashboards The metric library, Studio, collectors, and thresholds Vapi, Retell, ElevenLabs, Leaping, LiveKit, and custom Drive Roark from your terminal and CI pipelines Node.js, Python, and MCP Server REST API documentation # Bland AI Source: https://docs.roark.ai/documentation/integrations/bland Sync Bland AI agents and analyze voice AI conversations Live monitoring Voice simulations Chat simulations ## Overview The Bland AI integration syncs your voice agents and their calls into Roark. Agents and calls are pulled on a recurring schedule so every conversation is analyzed and evaluated automatically. *** ## Prerequisites Before setting up the integration, ensure you have: * A Bland AI account with active agents * A Bland AI API key (found in your [Bland AI dashboard](https://app.bland.ai)) * Call recording enabled in your Bland AI send call payload (see [Call Payload Configuration](#call-payload-configuration) below) *** ## Call Payload Configuration To sync calls with Roark, you must include two fields in your Bland AI send call payload: | Field | Type | Description | | :--------------- | :-------- | :------------------------------------------------------------------------------------------------------------------------------ | | **`persona_id`** | `string` | The agent/persona ID. Roark uses this to match calls to agents and determine which calls to sync based on your selected agents. | | **`record`** | `boolean` | Must be set to `true` so the call is recorded and Roark can access the audio for analysis. | Calls without `record: true` will not have audio available and cannot be synced into Roark. Calls without a `persona_id` will not be matched to any agent. *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **Bland AI** 3. Enter your configuration: | Field | Description | | :------------------- | :--------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your Bland AI API key | The system will validate your API key and fetch available agents. ### Step 2: Select Agents Choose which Bland AI agents to monitor: * **All Agents**: Sync calls from every agent in your account * **Selected Agents**: Choose specific agents to monitor Roark syncs calls based on the `persona_id` field in your call payload. Only calls matching your selected agents will be synced. ### Step 3: Activate Integration Review your settings and click **Create Integration** to begin syncing. *** ## What Gets Synced Bland AI integrations sync the following data: * **Calls**: Conversation audio, transcripts, and metadata * **Agents**: Agent/persona configurations * **Transcripts**: Full conversation transcripts with speaker labels * **Tool Calls**: Function/tool invocations during conversations * **Call Metadata**: Duration, status, participant information *** ## How Sync Works Bland AI uses **pull-based synchronization**: Roark periodically fetches new calls from the Bland AI API rather than receiving webhooks. | Phase | What Happens | | :------------- | :------------------------------------------------------------------------------------------------------------------------- | | **Agent Sync** | Fetches all agents, updates names and configurations | | **Call Sync** | Fetches conversations since last sync, downloads recordings, parses transcripts and tool invocations, creates call records | Roark matches incoming calls to agents using the `persona_id` included in your send call payload. Only calls with `record: true` and a valid `persona_id` are synced. *** ## Monitoring Integration Health Track your integration status in the Roark dashboard: * **Syncing**: Actively importing calls * **Active**: Running on schedule, pulling new calls periodically * **Paused**: Sync temporarily halted * **Error**: Configuration or connection issues View sync statistics including total calls synced, last sync timestamp, and job history. *** ## Agent Management Synced agents appear in: * **Simulation agent selection**: Use for testing * **Agent performance reports**: Track metrics per agent * **Comparison dashboards**: Analyze across agents Agent data updates automatically when modified in Bland AI. *** ## Next Steps Test synced agents with simulations Browse and analyze synced calls Automate metric collection on calls Explore other integrations # Custom Integrations Source: https://docs.roark.ai/documentation/integrations/custom-integrations Send calls from any platform using our API and SDKs ## Overview Not using one of our pre-built integrations? No problem. Send call data from any voice AI platform or custom application. Just provide an audio file and Roark handles the rest: transcription, speech analysis, sentiment detection, and metric collection. *** ## Send a Call Install the [Node.js SDK](/documentation/sdks/node-sdk): ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} npm install @roarkanalytics/sdk ``` ```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import Roark from '@roarkanalytics/sdk' const client = new Roark({ bearerToken: process.env.ROARK_API_BEARER_TOKEN, }) const call = await client.call.create({ recordingUrl: 'https://your-storage.com/call-recording.mp3', startedAt: '2024-01-15T10:00:00Z', interfaceType: 'PHONE', callDirection: 'INBOUND', agent: { name: 'Support Agent', customId: 'custom-agent-1', }, customer: { phoneNumberE164: '+1234567890', }, properties: { department: 'sales', region: 'us-east', }, }) ``` Install the [Python SDK](/documentation/sdks/python-sdk): ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} pip install roark-analytics ``` ```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import os from roark_analytics import Roark client = Roark( bearer_token=os.environ.get("ROARK_API_BEARER_TOKEN"), ) call = client.call.create( recording_url="https://your-storage.com/call-recording.mp3", started_at="2024-01-15T10:00:00Z", interface_type="PHONE", call_direction="INBOUND", agent={ "name": "Support Agent", "customId": "custom-agent-1", }, customer={ "phoneNumberE164": "+1234567890", }, properties={ "department": "sales", "region": "us-east", }, ) ``` Use the [REST API](/api-reference/introduction) directly: ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl --request POST \ --url https://api.roark.ai/v1/call \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "recordingUrl": "https://your-storage.com/call-recording.mp3", "startedAt": "2024-01-15T10:00:00Z", "interfaceType": "PHONE", "callDirection": "INBOUND", "agent": { "name": "Support Agent", "customId": "custom-agent-1" }, "customer": { "phoneNumberE164": "+1234567890" }, "properties": { "department": "sales", "region": "us-east" } }' ``` Once a call is created, it appears in [Call History](/documentation/observability/live-monitoring) and is automatically transcribed and analyzed. *** ## What You Can Send | Data | Description | | :----------------------- | :----------------------------------------------------------- | | **Audio file** | MP3, WAV, M4A, or FLAC (via public URL or base64) | | **Metadata** | Agent info, customer details, call direction, interface type | | **Custom properties** | Arbitrary key-value pairs for filtering and segmentation | | **Tool calls** | Function/tool invocations made during the conversation | | **Pre-transcribed text** | If you already have a transcript, send it along | ### Audio Requirements | Spec | Requirement | | :-------------- | :---------------------------------------- | | **Format** | MP3, WAV, M4A, or FLAC | | **Sample Rate** | 8kHz or higher | | **Channels** | Mono or stereo | | **Duration** | Up to 2 hours | | **File Size** | Max 500MB | | **Access** | Publicly accessible URL or base64 encoded | *** ## What Happens Next Once a call is ingested, Roark automatically: 1. **Transcribes and analyzes** the conversation: speech patterns, sentiment, 64+ emotions, interruptions, and more 2. **Runs metric collectors**: any active [metric collectors](/documentation/metrics/metric-collectors) collect metrics automatically (LLM evaluations, compliance checks, custom KPIs) 3. **Makes it searchable**: the call appears in [Call History](/documentation/observability/live-monitoring) with full filtering by properties, agent, duration, and more *** ## Use With Simulations Agents created via the API can also be used in [simulations](/documentation/simulation-testing/overview). When you send calls with an `agent.customId`, Roark creates or matches an agent record that you can then target in simulation run plans, useful for testing custom-built voice agents that aren't on a pre-built platform. *** ## MCP Server If you use an AI-powered development environment, the [Roark MCP Server](/documentation/sdks/mcp-server) gives your AI assistant direct access to the Roark API (including creating calls, querying metrics, and managing agents) without writing integration code manually. *** ## Getting Started Create an API key in your [Roark dashboard](/documentation/getting-started/api-keys) Upload an audio file using the SDK or REST API above Set up [metric collectors](/documentation/metrics/metric-collectors) to automatically collect metrics on every call Browse calls in [Call History](/documentation/observability/live-monitoring), build [reports](/documentation/observability/reports), and create [dashboards](/documentation/observability/dashboards) *** ## Resources Send OpenTelemetry traces from any platform Full SDK guide and reference Full SDK guide and reference Complete REST API documentation # ElevenLabs Source: https://docs.roark.ai/documentation/integrations/elevenlabs Sync ElevenLabs conversational AI agents and run voice and chat simulations Live monitoring Voice simulations Chat simulations ## Overview The ElevenLabs integration syncs your conversational AI agents and their calls into Roark. Agents, prompts, phone numbers, and tools are imported automatically, and calls are pulled on a recurring schedule so every conversation is analyzed. A single ElevenLabs integration unlocks all three of Roark's capabilities for connected agents: * **Live call monitoring**: pull conversation audio, transcripts, and metadata on a recurring schedule. * **Voice simulations**: drive synthesized voice conversations against synced agents over phone or WebRTC. * **Chat simulations**: drive [text-based simulations](/documentation/simulation-testing/chat-simulations) against synced agents over the ElevenLabs Conversational AI WebSocket. Roark auto-creates a chat endpoint for each agent on integration setup, so once you've connected ElevenLabs you can use any synced agent in a chat run plan immediately. *** ## Prerequisites Before setting up the integration, ensure you have: * An ElevenLabs account with active conversational AI agents * An ElevenLabs API key (found in your ElevenLabs dashboard) * Agents with **voice recording enabled** (agents with Zero Retention Mode or recording disabled cannot sync calls) *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **ElevenLabs** 3. Enter your configuration: | Field | Description | | :------------------- | :--------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your ElevenLabs API key | The system will validate your API key and fetch available agents. ### Step 2: Configure Historical Sync Choose how far back to import existing calls: * **Default**: Last 90 days of call history * **Custom**: Select a specific date range * **Skip**: Only sync new calls going forward Historical sync runs as a backfill job during initial setup. Ongoing calls are synced automatically on a recurring schedule. ### Step 3: Select Agents Choose which ElevenLabs agents to monitor: * **All Agents**: Sync calls from every agent in your account * **Selected Agents**: Choose specific agents to monitor The agent selector shows each agent's name and ID. Agents with Zero Retention Mode enabled or voice recording disabled will be flagged as unavailable for call sync. ### Step 4: Activate Integration Review your settings and click **Create Integration** to begin syncing. *** ## What Gets Synced ElevenLabs integrations sync comprehensive data: * **Calls**: Conversation audio, transcripts, and metadata * **Agents**: Agent configurations and settings * **Prompts**: System prompts configured for each agent * **Phone Numbers**: Phone number endpoints assigned to agents * **Tools**: Tool definitions available to agents * **Call Metadata**: Duration, direction (inbound/outbound), interface type (phone/web), termination reason *** ## How Sync Works ElevenLabs uses **pull-based synchronization**: Roark periodically fetches new calls from the ElevenLabs API rather than receiving webhooks. | Phase | What Happens | | :--------------- | :---------------------------------------------------------------------------------------------------- | | **Agent Sync** | Fetches all agents, updates names and configurations, syncs prompts, phone numbers, and tools | | **Health Check** | Verifies each agent's recording and privacy settings are compatible | | **Call Sync** | Fetches conversations since last sync, downloads audio, parses tool invocations, creates call records | Calls must be at least 10 seconds long and have a successful status to be synced. *** ## Agent Privacy Settings ElevenLabs agents have privacy controls that affect what Roark can sync: * **Zero Retention Mode** enabled → Call recordings unavailable, calls cannot be synced * **Record Voice** disabled → Same as above During health checks, Roark flags agents with these settings so you know which agents can and cannot sync call data. *** ## Monitoring Integration Health Track your integration status in the Roark dashboard: * **Syncing**: Actively importing calls * **Active**: Running on schedule, pulling new calls periodically * **Paused**: Sync temporarily halted * **Error**: Configuration or connection issues View sync statistics including total calls synced, last sync timestamp, and job history. *** ## Next Steps Test synced agents with simulations Browse and analyze synced calls Automate metric collection on calls Explore other integrations # Exporting to Snowflake Source: https://docs.roark.ai/documentation/integrations/exporting-to-snowflake Sync your Roark call analysis into a data warehouse to join it against your own operational data ## Overview Roark analyzes every call and produces structured metric values (frustration, sentiment, task completion, latency, custom metrics, and more). A common next step is to land that analysis in your own data warehouse so you can join it against your operational data, for example correlating call frustration with downstream business outcomes. This guide covers the recommended pattern for syncing Roark analysis into **Snowflake** using the tools available today. The same approach works for BigQuery, Redshift, or Databricks: only the load step changes. Want a fully managed connector or a scheduled file drop into your own bucket? See [Deeper integrations](#deeper-integrations) at the bottom of this page and [reach out](/documentation/resources/support). We are actively expanding warehouse-export options. *** ## The recommended pattern The pattern is **event-driven pull**: Roark notifies you when a call finishes analysis, you pull the results from the API, land the raw JSON in cloud storage, and load it into Snowflake. ``` Roark webhook ──▶ your handler ──▶ fetch metrics/transcript ──▶ S3 (raw JSON) ──▶ Snowpipe ──▶ Snowflake ``` Roark webhooks are a **notification, not the payload**. When a call is analyzed you receive the `callId`, not the metric values themselves, so your handler makes one follow-up API call to fetch the analysis. This keeps payloads small and lets you pull exactly the fields you need. Generate a key scoped to the project you want to export. Read endpoints require the `call:read`, `metric:read`, and `transcript:read` permissions. See [API Keys](/documentation/getting-started/api-keys). Add a webhook endpoint and subscribe to `call.analysis.completed`. See [Webhooks](/documentation/integrations/webhooks). In your handler, call the metrics (and optionally transcript) endpoints for the `callId` you received. Write each call's payload to S3 (or GCS / Azure Blob). Store it as raw JSON so schema changes never break ingestion. Point Snowpipe at the bucket to auto-ingest, then flatten into modeled tables. *** ## What you can export All data is served from the Customer API at `https://api.roark.ai/v1` and is scoped to the project your API key belongs to. | Data | Endpoint | Permission | | ---------------------------------------- | ------------------------------------------------------ | ----------------- | | List of calls (paginated) | `GET /v1/call` | `call:read` | | Single call detail | `GET /v1/call/{callId}` | `call:read` | | Metric values for a call | `GET /v1/call/{callId}/metrics` | `metric:read` | | Transcript for a call | `GET /v1/call/{callId}/transcript` | `transcript:read` | | Metric values for a whole collection job | `GET /v1/metric/collection-jobs/{jobId}/metric-values` | `metric:read` | | Metric definitions (catalog) | `GET /v1/metric/definitions` | `metric:read` | Chats have the identical set of endpoints under `/v1/chat`. Every call carries an `externalId` (your own correlation ID, set when the call is ingested) and a `properties` map of custom key-values. These are your join keys back to operational data in your warehouse. *** ## Approach A: Event-driven (recommended for ongoing sync) Best for keeping the warehouse continuously fresh with low latency. ```python Python (webhook handler) theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} @app.route('/webhooks/roark', methods=['POST']) def handle_webhook(): payload = request.json if payload['event'] == 'call.analysis.completed': call_id = payload['data']['callId'] # Pull the analysis Roark just finished call = get(f'/v1/call/{call_id}') metrics = get(f'/v1/call/{call_id}/metrics') # Land raw JSON in S3, keyed by call id s3.put_object( Bucket='my-roark-export', Key=f'calls/{call_id}.json', Body=json.dumps({'call': call, 'metrics': metrics}), ) return '', 200 def get(path): r = requests.get( f'https://api.roark.ai{path}', headers={'Authorization': f'Bearer {ROARK_API_KEY}'}, timeout=10, ) r.raise_for_status() return r.json() ``` ```typescript TypeScript (webhook handler) theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} app.post('/webhooks/roark', async (req, res) => { const { event, data } = req.body if (event === 'call.analysis.completed') { const callId = data.callId const call = await get(`/v1/call/${callId}`) const metrics = await get(`/v1/call/${callId}/metrics`) await s3.putObject({ Bucket: 'my-roark-export', Key: `calls/${callId}.json`, Body: JSON.stringify({ call, metrics }), }) } res.sendStatus(200) }) const get = async (path: string) => { const r = await fetch(`https://api.roark.ai${path}`, { headers: { Authorization: `Bearer ${process.env.ROARK_API_KEY}` }, }) if (!r.ok) throw new Error(`${path} -> ${r.status}`) return r.json() } ``` *** ## Approach B: Scheduled backfill / poll Best for the initial backfill of historical data, or as a simpler alternative to webhooks. Page through the calls list on a schedule and fetch metrics per call. The list endpoint uses keyset (cursor) pagination. Sort by `startedAt` and walk the cursor to move forward in time. ```python Python (paginated pull) theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} def export_all_calls(): cursor = None while True: params = {'limit': 100, 'sortBy': 'startedAt', 'sortDirection': 'asc'} if cursor: params['after'] = cursor page = get('/v1/call', params=params) for call in page['data']: metrics = get(f"/v1/call/{call['id']}/metrics") write_to_s3(call, metrics) if not page['hasMore']: break cursor = page['nextCursor'] ``` ```bash cURL (single page) theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} curl -G "https://api.roark.ai/v1/call" \ -H "Authorization: Bearer $ROARK_API_KEY" \ --data-urlencode "limit=100" \ --data-urlencode "sortBy=startedAt" \ --data-urlencode "sortDirection=asc" \ --data-urlencode "after=$CURSOR" ``` For a large backfill you can also pull metric values a whole job at a time with `GET /v1/metric/collection-jobs/{jobId}/metric-values`, which returns every metric value for that collection job in one paginated stream rather than one request per call. *** ## Loading into Snowflake Land the raw JSON in an external stage, ingest into a `VARIANT` staging table, then flatten into modeled tables with `LATERAL FLATTEN`. Storing the raw payload first means a new metric or field never breaks your pipeline. ```sql theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} create or replace stage roark_stage url = 's3://my-roark-export/calls/' storage_integration = my_s3_integration file_format = (type = json); ``` ```sql theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} create table roark_raw (raw variant, loaded_at timestamp_ntz default current_timestamp()); create or replace pipe roark_pipe auto_ingest = true as copy into roark_raw (raw) from @roark_stage; ``` ```sql theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} create or replace view roark_call_metrics as select raw:call:id::string as call_id, raw:call:externalId::string as external_id, -- your join key raw:call:startedAt::timestamp as started_at, raw:call:durationMs::number as duration_ms, raw:call:callDirection::string as direction, m.value:slug::string as metric_slug, m.value:name::string as metric_name, m.value:type::string as metric_type, m.value:value::variant as metric_value, m.value:captureStatus::string as capture_status from roark_raw, lateral flatten(input => raw:metrics:data) m; ``` ```sql theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} -- Does call frustration predict whether a caregiver takes a shift? select f.metric_value::float as frustration, s.shift_accepted from roark_call_metrics f join shifts s on s.roark_external_id = f.external_id where f.metric_slug = 'frustration'; ``` Model the flattened view once per metric type you care about, or keep one long-format `(call_id, metric_slug, metric_value)` table and pivot in your BI layer. Long format handles new metrics with zero schema changes. *** ## Things to know * **Idempotency.** Keying S3 objects and Snowflake rows by `callId` (plus the metric collection job id) makes re-delivery and backfills safe to re-run without double counting. * **Webhook delivery.** Roark retries failed deliveries up to 3 times with backoff. Respond `200` within 5 seconds and process asynchronously. See [Webhooks](/documentation/integrations/webhooks). * **Date filtering.** The calls list does not yet accept an explicit `from`/`to` date range. Approximate a range by sorting on `startedAt` and walking the cursor until you pass your window. * **Transcripts** are fetched per call from the transcript endpoint; they are not included in the metrics payload. * **Rate limits and pagination.** Use `limit=100` and honor `nextCursor`/`hasMore` rather than assuming a fixed page count. *** ## Deeper integrations The pattern above uses only what is available today. If you need something more managed, we are building toward it and would like to scope it with you: A Fivetran / Airbyte-style connector with incremental, updated-since cursors so your warehouse stays in sync automatically. Roark writes incremental Parquet or CSV to your own S3 bucket on a schedule, ready for Snowpipe. A hands-off Snowflake Secure Data Share so there is nothing to run on your side. Tell us your volume, latency needs, and the joins you want to run. *** ## Next steps Set up the analysis-completed notification Create a read-only key for the export Full endpoint and parameter reference Analyze the same data inside Roark # Google CES Source: https://docs.roark.ai/documentation/integrations/google-ces Run chat simulations against Google Customer Engagement Suite apps Chat simulations Live monitoring Voice simulations ## Overview The Google CES integration connects a [Google Customer Engagement Suite](https://docs.cloud.google.com/customer-engagement-ai/) app to Roark for [chat simulations](/documentation/simulation-testing/chat-simulations). Roark sends each persona turn to the CES [`runSession` endpoint](https://docs.cloud.google.com/customer-engagement-ai/conversational-agents/ps/reference/rest/v1/projects.locations.apps.sessions/runSession) and records the agent's reply as part of the chat transcript. Authentication uses [Workload Identity Federation (WIF)](https://docs.cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds) so **no long-lived service account keys ever leave your GCP tenant**. Roark mints a short-lived access token at simulation time using credentials your team controls and can revoke at any time. *** ## Prerequisites Before getting started, you'll need: * A Google Cloud project with a published Customer Engagement Suite app * Permissions in that project to create a Workload Identity Pool and configure a service account with access to your CES app * The CES app coordinates: **Location**, **App ID**, **Version ID**, **Deployment ID** *** ## Setting Up Workload Identity Federation WIF setup requires a few values from Roark to authorize in your Workload Identity Pool. **Reach out to us and we'll walk you through it end-to-end**: typically a single back-and-forth to share the values you need, after which you generate the WIF credential JSON on your side and paste it into Roark. Email **[support@roark.ai](mailto:support@roark.ai)** and we'll get you set up. Include your GCP project ID and CES app coordinates if you have them handy. Once WIF is configured on your side, you'll have a credential configuration JSON document (`type: "external_account"`) ready to paste into Roark. *** ## Creating the Integration in Roark Navigate to **Agents**, click **Connect Agent** in the top right, choose **Existing Platform**, and select **Google CES**. | Field | Description | | :-------------------- | :---------------------------------------------------------------------------------------------------- | | **Integration Name** | Friendly name for this integration | | **WIF Config JSON** | The full external-account credential JSON from your Workload Identity Pool. Stored encrypted at rest. | | **CES Location** | GCP region for your CES app (e.g. `us`, `global`) | | **CES App ID** | UUID of the CES app, from the CES console | | **CES Version ID** | UUID of the app version to run | | **CES Deployment ID** | UUID of the deployment to target | On save, Roark validates the WIF document's shape, stores it encrypted, and auto-provisions: * One **Roark agent** named after the integration * One **`GOOGLE_CES` chat endpoint** linked to the integration The endpoint is immediately usable in run plans. *** ## How Authentication Works at Runtime When a chat simulation runs against a Google CES endpoint, Roark mints a fresh access token for the request rather than storing one: 1. Roark uses your WIF configuration to obtain a federated token from Google STS. 2. The federated token is exchanged for a short-lived OAuth access token that impersonates your configured service account. 3. Roark sends `Authorization: Bearer ` on each call to `…/sessions/{sessionId}:runSession`. Tokens are short-lived (≈1 hour) and live only in memory for the duration of the simulation. You can revoke access at any time by removing Roark from your Workload Identity Pool. *** ## How Conversations Are Driven Each simulation generates a unique session ID per run. For every persona turn, Roark POSTs to: ``` POST https://-ces.googleapis.com/v1beta/projects//locations//apps//sessions/:runSession ``` with a body referencing your `app_version` and `deployment` plus the user input text. The agent's reply is read from `outputs[0].text` and added to the chat transcript. CES creates the session on the first call with a new session ID and resumes it on subsequent calls with the same ID, so no separate bootstrap call is needed. *** ## What's Not Supported * **Voice simulations**: Google CES integrations are chat-only. * **Live monitoring / call import**: Roark does not pull historical or live conversations from CES. Use chat simulations to evaluate the agent. *** ## Next Steps Test your CES app with persona-driven chats Define the conversations to test Automate evaluation on every chat Explore other integrations # IP Whitelisting Source: https://docs.roark.ai/documentation/integrations/ip-whitelisting Whitelist Roark Analytics production IPs for firewalls, webhooks, and presigned URLs If you need to whitelist Roark Analytics' IP addresses for accessing your infrastructure (for example, presigned URLs or private APIs), use the following static IP addresses. *** ## Production Environment All outbound requests from Roark Analytics production services originate from these NAT gateway IPs: * 54.175.209.200 * 54.161.212.170 Important: You must whitelist both IP addresses. Traffic may route through either one depending on availability zone placement. *** ## Use Cases Whitelist these IPs when: * Providing presigned URLs for audio files or recordings * Exposing webhook endpoints with IP restrictions * Configuring firewall rules for API integrations * Setting up private API access *** ## Region These IPs are for the US-EAST-1 region where Roark Analytics production services are hosted. # Kore AI Source: https://docs.roark.ai/documentation/integrations/kore Run chat simulations against Kore AI Agent Platform apps Chat simulations Live monitoring Voice simulations ## Overview The Kore AI integration connects an app on the [Kore AI Agent Platform](https://docs.kore.ai/agent-platform/) to Roark for [chat simulations](/documentation/simulation-testing/chat-simulations). Roark talks to your app over the Agent Platform v2 HTTP API, bootstrapping a session, sending each persona turn through `/runs/execute`, and tearing the session down when the conversation ends. *** ## Prerequisites Before connecting, you'll need from the Kore AI Agent Platform: * An **App ID** (UUID) for the agent app you want to test * An **Environment name** the app is deployed to (e.g. `production`) * An **API key** (`x-api-key`) with permission to create sessions and execute runs against that app * The **base URL** of your Kore tenant, if you're on a non-default or self-hosted Kore domain. Most customers can leave this blank and Roark will use `https://agent-platform.kore.ai/api/v2`. See the [Kore AI Agent Platform API reference](https://docs.kore.ai/agent-platform/apis/agentic-apps/overview/) for where to find these in the Kore console. *** ## Creating the Integration in Roark Navigate to **Agents**, click **Connect Agent** in the top right, choose **Existing Platform**, and select **Kore AI**. | Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your Kore `x-api-key`. Stored encrypted at rest. | | **App ID** | UUID of the Kore agent app | | **Environment** | Environment name the app is deployed to (e.g. `production`) | | **Base URL** *(optional)* | Override the Kore API base URL for enterprise or self-hosted deployments. Defaults to `https://agent-platform.kore.ai/api/v2`. | On save, Roark stores the credentials encrypted and auto-provisions: * One **Roark agent** named after the integration * One **`KORE` chat endpoint** linked to the integration The endpoint is immediately usable in run plans. Credentials never leave the integration. The endpoint references them by ID. In edit mode, leave the **API Key** field blank to keep the existing key. Only fill it in if you want to rotate the credential. *** ## How Conversations Are Driven Each chat simulation runs through a three-leg session lifecycle against your Kore app: | Phase | Request | | :------------ | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Bootstrap** | `POST {baseUrl}/apps/{appId}/environments/{env}/sessions`: creates a session and returns a session reference. | | **Per turn** | `POST {baseUrl}/apps/{appId}/environments/{env}/runs/execute`: sends the persona's message as text input under the bootstrapped session, and reads the agent's reply from `outputs[0].content`. | | **Teardown** | `POST {baseUrl}/apps/{appId}/environments/{env}/sessions/terminate`: closes the session when the chat ends. | All requests are sent with your `x-api-key` in the header. The API key is decrypted in memory only for the lifetime of the chat session and is never logged. *** ## What's Not Supported * **Voice simulations**: Kore AI integrations are chat-only in Roark. * **Live monitoring / call import**: Roark does not pull historical conversations from Kore. Chat simulations are the way to evaluate your Kore app's behavior in Roark. *** ## Next Steps Test your Kore app with persona-driven chats Define the conversations to test Automate evaluation on every chat Explore other integrations # Leaping Source: https://docs.roark.ai/documentation/integrations/leaping Sync Leaping AI agents and analyze voice conversations Live monitoring Voice simulations Chat simulations ## Overview The Leaping integration syncs your Leaping AI agents and their calls into Roark. Agents, prompts, and tools are imported automatically, and calls are pulled on a recurring schedule so every conversation is analyzed. *** ## Prerequisites Before setting up the integration, ensure you have: * A Leaping AI account with active agents * A Leaping API key (found in your Leaping dashboard) * At least one configured agent in Leaping *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **Leaping** 3. Enter your configuration: | Field | Description | | :------------------- | :--------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your Leaping API key | The system will validate your API key and fetch available agents. ### Step 2: Configure Historical Sync Choose how far back to import existing calls: * **Default**: Last 90 days of call history * **Custom**: Select a specific date range * **Skip**: Only sync new calls going forward Historical sync runs as a backfill job during initial setup. Ongoing calls are synced automatically on a recurring schedule. ### Step 3: Select Agents Choose which Leaping agents to monitor: * **All Agents**: Sync calls from every agent in your account * **Selected Agents**: Choose specific agents to monitor The agent selector shows each agent's name and ID. ### Step 4: Activate Integration Review your settings and click **Create Integration** to begin syncing. *** ## What Gets Synced Leaping integrations sync comprehensive data: * **Calls**: Conversation transcripts, metadata, and call events * **Agents**: Agent configurations and endpoints * **Prompts**: System prompts configured for each agent * **Tools**: Tool definitions available to agents * **Call Metadata**: Duration, status, properties, and events *** ## How Sync Works Leaping uses **pull-based synchronization**: Roark periodically fetches new calls from the Leaping API rather than receiving webhooks. | Phase | What Happens | | :--------------- | :------------------------------------------------------------------------------- | | **Agent Sync** | Fetches agents and endpoints, updates configurations, syncs prompts and tools | | **Health Check** | Verifies API connectivity and agent availability | | **Call Sync** | Fetches calls since last sync for each agent, processes transcripts and metadata | Calls with status `completed`, `transferred`, or `dropped` are synced. *** ## Monitoring Integration Health Track your integration status in the Roark dashboard: * **Syncing**: Actively importing calls * **Active**: Running on schedule, pulling new calls periodically * **Paused**: Sync temporarily halted * **Error**: Configuration or connection issues View sync statistics including total calls synced, last sync timestamp, and job history. *** ## Next Steps Test synced agents with simulations Browse and analyze synced calls Automate metric collection on calls Explore other integrations # LiveKit Cloud Source: https://docs.roark.ai/documentation/integrations/livekit Connect LiveKit Cloud for real-time voice communication monitoring Live monitoring Voice simulations Chat simulations ## Overview The LiveKit Cloud integration enables real-time monitoring of voice communications built on [LiveKit Cloud](https://cloud.livekit.io/). Roark connects to your LiveKit project over webhooks and the server API (no changes to your agent code) and streams call data in for analysis, evaluation, and quality monitoring. Running your own `livekit-server` instead of LiveKit Cloud? Webhooks and the dashboard-managed flow described here still work, but the recommended path for self-hosted deployments is the drop-in Python SDK. See [LiveKit (self-hosted)](/documentation/integrations/livekit-self-hosted). *** ## Prerequisites Before setting up the integration, ensure you have: * A [LiveKit Cloud](https://cloud.livekit.io/) account and project * LiveKit API credentials (API Key and Secret) * Admin access to configure webhooks in LiveKit *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **LiveKit** 3. Enter the following configuration: | Field | Description | Example | | :------------------- | :--------------------------------- | :---------------------------- | | **Integration Name** | Friendly name for this integration | Production LiveKit | | **Server URL** | Your LiveKit WebSocket URL | `wss://example.livekit.cloud` | | **API Key** | LiveKit API key | `APIxxxxxxxxxxxxx` | | **API Secret** | LiveKit API secret | Keep this secure | ### Step 2: Configure Webhook After creating the integration, Roark provides a webhook URL: 1. Copy the webhook URL from the integration settings 2. Navigate to your LiveKit dashboard 3. Go to **Settings → Webhooks** 4. Add the Roark webhook URL 5. Select the following events: * `room_started` * `room_finished` * `participant_joined` * `egress_ended` ### Step 3: Configure Agents Under the **Agents** section of the integration, decide how LiveKit calls map to Roark agents: * **Import named LiveKit agents**: Roark fetches the named agents configured in your LiveKit project so you can choose which ones to track. Each imported agent becomes its own Roark agent, and calls are routed based on the LiveKit agent name that participated in the room. * **Assign all calls to a single Roark agent**: Pick one Roark agent and every call from this integration is attributed to it. Use this when you rely on LiveKit's automatic dispatch (unnamed agents) or when you don't care about distinguishing multiple agents. You can switch between these modes later, but existing calls will keep the agent attribution they had at ingest time. ### Step 4: WebRTC Endpoints (optional) The **WebRTC** section of the integration page is where you manage all LiveKit WebRTC endpoints for this integration. On first setup it can auto-create one endpoint per imported agent so you can start running simulations immediately; after the integration exists you can come back to this same section at any time to add, edit, or remove endpoints. See [WebRTC Endpoints](#webrtc-endpoints) below for the full list of settings. You can skip this step and add endpoints later if you prefer. ### Step 5: Configure Metric Collectors Calls synced from LiveKit are scored by your [metric collectors](/documentation/metrics/metric-collectors). Set them up globally under **Metrics → Collectors**, or scope collectors to a specific agent from that agent's **Collectors** section. ### Step 6: Activate Integration Toggle the integration status to **Active** to begin receiving call data. *** ## WebRTC Endpoints Roark can run simulations directly over WebRTC against a LiveKit room instead of placing a phone call. This is the recommended path for LiveKit agents because it avoids SIP trunking and exercises the exact transport your production users connect over. ### Managing WebRTC Endpoints All LiveKit WebRTC endpoints for an integration are managed from the **WebRTC** section of the integration page: 1. Go to **Agents**, expand the connected sources at the top of the page, and choose your **LiveKit** source. The integration form opens on the right. 2. Expand the **WebRTC** section to see every endpoint attached to an agent that belongs to this integration. Each row shows the agent name and environment. 3. Click an existing endpoint to edit it, or click **Add WebRTC endpoint** to create a new one. The add flow opens the same editor used on the agent page and is scoped to agents managed by this integration. On initial integration setup, Roark can also auto-create one WebRTC endpoint per imported agent in a single step. See [Step 4: WebRTC Endpoints](#step-4-webrtc-endpoints-optional) above. After the integration exists, use the flow described here for every change. ### Room Management Mode When configuring the endpoint you pick who is responsible for creating the LiveKit room that a simulation runs in. Roark creates the room in your LiveKit project at the start of each simulation using the API credentials from the integration, then joins the room as the simulated caller. * **Agent dispatch**: If a **Named LiveKit Agent** is set on the integration, Roark explicitly dispatches that agent into the room so it joins as soon as the room is created. If no named agent is set, Roark relies on LiveKit's [automatic dispatch](https://docs.livekit.io/agents/worker/agent-dispatch/) behavior: any unnamed worker connected to your LiveKit project will pick up the room automatically. * **Room metadata**: Optionally provide a JSON string that Roark attaches to the room it creates. Use this to pass context to your agent (for example, a customer ID, scenario tag, or feature flag) that your worker can read from the room metadata on join. This mode is the easiest to set up and works for most agents that use LiveKit's standard dispatch model. Provide an HTTPS URL that Roark calls when a simulation starts. Your endpoint is responsible for creating the LiveKit room, dispatching your agent into it, and returning the room name so Roark can join. **Request**: Roark sends an HTTPS request to your URL when a simulation starts. **Expected response**: Return a JSON body containing the `roomName` of the LiveKit room you created. Roark then connects to the same LiveKit server and joins that room as the simulated caller. ```json theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} { "roomName": "sim-session-abc123" } ``` Use this mode when you need custom room setup logic: for example, if you provision rooms through your own backend, need to wire up per-session tokens or tools, or run a bespoke dispatch flow that Roark's auto-create path can't express. Once the endpoint exists, use it like any other agent endpoint when building [run plans](/documentation/simulation-testing/run-plans) and [running simulations](/documentation/simulation-testing/running-simulations). *** ## What Gets Synced LiveKit integrations sync the following data: * **Call Events** - Room creation, participant joining, call completion * **Recording Files** - MP4 recordings from egress events * **Call Metadata** - Duration, participant count, room configuration * **Transcripts** - When speech-to-text is enabled * **Named Agents** - When agent import is enabled in the [Agents](#step-3-configure-agents) step, the list of named LiveKit agents from your project LiveKit does not sync agent prompts or worker source code, as it's a real-time communication platform rather than an AI agent provider. Only agent names are imported so Roark can attribute calls to the correct agent. *** ## Webhook Events The integration processes these LiveKit events: | Event | Description | Roark Action | | :------------------- | :-------------- | :-------------------------------- | | `room_started` | Room created | Initialize call record | | `room_finished` | Room ended | Finalize call, trigger evaluation | | `participant_joined` | User joined | Update participant list | | `egress_ended` | Recording ready | Process recording file | *** ## Call Sampling & Metadata Filtering Roark provides two ways to control which LiveKit calls are processed: ### Random Sampling Set a **sampling rate** (0–100%) in the integration settings to randomly sample a percentage of calls. This is useful when you want to evaluate a representative subset without any code changes. ### Metadata Filtering (Per-Call Control) For more granular control, set `roark.skip` to `true` in your LiveKit room metadata to tell Roark to skip processing for specific calls. ```python Python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} from livekit import api room = api.CreateRoomRequest( name="my-room", metadata='{"roark.skip": true}' ) ``` ```typescript TypeScript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import { RoomServiceClient } from 'livekit-server-sdk'; const roomService = new RoomServiceClient(url, apiKey, apiSecret); await roomService.createRoom({ name: 'my-room', metadata: JSON.stringify({ 'roark.skip': true }), }); ``` **How it works:** * Calls **without** `roark.skip` in metadata are processed normally (safe default) * Calls **with** `roark.skip` set to `true` are skipped entirely: no recording, no evaluation, no processing * The check happens at the webhook level, so skipped calls use zero Roark resources * You can combine metadata filtering with sampling: sampling is applied first, then metadata filtering Metadata filtering is ideal when you want deterministic control over which calls are processed, such as skipping internal test calls or only evaluating calls for specific customers. *** ## Monitoring Integration Health Check your integration status in the Roark dashboard: * **Active** 🟢 - Receiving and processing events * **Inactive** 🔴 - Integration paused * **Error** ⚠️ - Configuration or connection issues View recent webhook deliveries and any error messages in the integration details. *** ## Next Steps Send OpenTelemetry traces from your LiveKit agent Define metrics and pass/fail thresholds Monitor calls in real-time Learn about all integrations Instrument your own livekit-server with the Python SDK # LiveKit (self-hosted) Source: https://docs.roark.ai/documentation/integrations/livekit-self-hosted Instrument a self-hosted LiveKit Agents worker with the roark-analytics[livekit] SDK Live monitoring Voice simulations Chat simulations ## Overview The self-hosted LiveKit integration monitors voice agents built on the open-source [LiveKit Agents](https://docs.livekit.io/agents/) framework that run against your own LiveKit server. Drop the [`roark-analytics[livekit]`](https://pypi.org/project/roark-analytics/) helper into your agent entrypoint and call lifecycle, transcripts, tool calls, and a recording are forwarded to Roark automatically. No other code changes required. The SDK talks only to Roark's own API; it never connects to your LiveKit server. Using [LiveKit Cloud](https://cloud.livekit.io/)? You can use this SDK there too, but the dashboard-managed webhook integration is usually simpler. See [LiveKit Cloud](/documentation/integrations/livekit). *** ## Prerequisites * A LiveKit Agents worker (Python 3.10+, `livekit-agents` 1.x) * Your own LiveKit server * A Roark API key with **WRITE** scope ([generate one](/documentation/getting-started/api-keys)) *** ## Setup Instructions ### Step 1: Install the SDK ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} pip install "roark-analytics[livekit]" ``` ### Step 2: Configure your API key The only setting you need to provide is your Roark API key. ```bash title=".env" theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} # Roark API key: create one on the API keys page in your Roark project. ROARK_API_KEY=rk_live_replace_me ``` | Variable | Required | Purpose | | :---------------------------- | :------- | :--------------------------------------------------------------------------------- | | `ROARK_API_KEY` | Yes | Roark API key with WRITE scope. Pass it to the helper as `api_key=`. | | `ROARK_OBSERVABILITY_ENABLED` | No | Set to `false` to make `observe_session` a no-op. See [Kill switch](#kill-switch). | ### Step 3: Wire `observe_session` into your entrypoint Call `observe_session(...)` **before `session.start()`** so the audio taps are installed before the session begins streaming frames. That single call wires up everything: ```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} import os from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli from livekit.plugins import cartesia, openai, silero, speechmatics from roark_analytics_python_livekit import observe_session SYSTEM_PROMPT = "You are a friendly voice assistant. Keep replies short. They are spoken aloud." class Assistant(Agent): def __init__(self) -> None: super().__init__(instructions=SYSTEM_PROMPT) async def entrypoint(ctx: JobContext) -> None: # Connect first: Roark keys the call on the LiveKit room sid (RM_…), # which is also what links OpenTelemetry traces to the call. await ctx.connect() session = AgentSession( stt=speechmatics.STT(), llm=openai.LLM(model="gpt-4o-mini"), tts=cartesia.TTS(), vad=silero.VAD.load(), ) # --- Roark analytics --------------------------------------------------- # Must be called BEFORE session.start(). Failures are logged and # swallowed: your agent keeps running even if Roark is unreachable. await observe_session( ctx, session, api_key=os.environ["ROARK_API_KEY"], agent_id="support-bot-v1", agent_name="Support Bot", agent_prompt=SYSTEM_PROMPT, ) # ----------------------------------------------------------------------- await session.start(room=ctx.room, agent=Assistant()) if __name__ == "__main__": cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint)) ``` That's the full integration: agent registration, transcripts, tool calls, and the recording are all handled by the helper. | Argument | Required | Description | | :------------- | :------- | :----------------------------------------------------------------------------------------- | | `ctx` | Yes | The `JobContext` passed to your agent entrypoint. | | `session` | Yes | The `AgentSession` you're about to `start()`. | | `api_key` | Yes | Your Roark API key (WRITE scope). | | `agent_id` | Yes | Stable, customer-defined agent identifier. Used for lazy registration. | | `agent_name` | No | Display name shown in the Roark dashboard. | | `agent_prompt` | No | System prompt: persisted as the agent's prompt revision, so changes are tracked over time. | | `**metadata` | No | Free-form metadata forwarded on `call-started` for Roark-side correlation. | ### Step 4: Verify the connection Run your worker against your LiveKit server and place a test call: ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} python agent.py dev ``` As the call runs you should see Roark log lines, then the recording upload begin after a few seconds of audio: ``` tapping user audio input (...) tapping agent audio output (...) call-started: ... ``` Within a few seconds you should also see, in your Roark dashboard: 1. The agent appear under the **LiveKit** source filter 2. The call appear in the calls table with status `In Progress` 3. After the call ends, the transcript, tool invocations, and recording attached to the call *** ## How It Works `observe_session` subscribes to the standard `AgentSession` event surface and ships a compact event timeline to Roark: | Phase | Source | What's captured | | :---------------- | :-------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------ | | **Session start** | `JobContext.connect()` | `call-started`, keyed on the LiveKit room sid (`RM_…`). Agent is lazy-registered the first time Roark sees an `agent_id`. | | **Transcripts** | `conversation_item_added` | Message role + content, for both user and assistant turns. | | **Tool calls** | `function_tools_executed` | Paired `tool_call` / `tool_result` records, keyed by `tool_call_id`. | | **Recording** | Taps on `session.input.audio` (user) + `session.output.audio` (agent) | Audio streamed to Roark during the call. | | **Session end** | `ctx.add_shutdown_callback(...)` | Flushes pending state, drains in-flight uploads, POSTs `call-ended`. | Failures are logged and swallowed: **the helper never raises into your session**. Your agent keeps running even if Roark is unreachable. *** ## What Gets Synced * **Calls**: Lifecycle with timing and end reason * **Agents**: Lazy-registered on first sight using the `agent_id` / `agent_name` you pass * **Prompts**: System prompt captured at call start as a prompt revision * **Transcripts**: Per-turn messages with role, content, and timestamp * **Tool Invocations**: Tool call IDs, names, JSON arguments, and results * **Recordings**: Audio recording of the call Roark only sees what the helper forwards. If you remove `observe_session` from a worker (or set `ROARK_OBSERVABILITY_ENABLED=false`), no data flows for those calls. *** ## Agent Management LiveKit agents are **lazy-registered** the first time the helper reports them: * The first `call-started` event with a new `agent_id` creates the agent in Roark * Subsequent events update the agent's name and prompt if they change * These agents appear on the agents page under the **LiveKit** source filter Once an agent exists, it can be used in [simulations](/documentation/simulation-testing/running-simulations), [run plans](/documentation/simulation-testing/run-plans), and [agent reports](/documentation/observability/reports) just like agents from any other provider. *** ## Kill Switch Disable instrumentation at runtime without touching code. Set `ROARK_OBSERVABILITY_ENABLED=false` to make `observe_session` a no-op (it returns `None`). Values treated as off: `false`, `0`, `no`, `off` (case-insensitive); anything else (or the variable being absent) keeps it enabled. *** ## Troubleshooting The helper registers a shutdown callback on the `JobContext`, which fires when LiveKit ends the job. If your transport tears down without firing the hook, call `await state.aflush(reason="...")` explicitly from your own disconnect handler: `aflush()` is idempotent, so the normal shutdown path will no-op if both fire. ```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} state = await observe_session(ctx, session, api_key=..., agent_id=...) # ... later, from your disconnect handler: await state.aflush(reason="client-disconnected") ``` Transcripts come from the `conversation_item_added` event on `AgentSession`. If you use a custom pipeline that bypasses `AgentSession.start(room=…, agent=…)`, that event may never fire. Verify by adding your own `session.on("conversation_item_added", print)` listener. The user side is tapped from `session.input.audio`; the agent side from `session.output.audio`. Because the taps are installed by `observe_session`, it **must be called before `session.start()`**. Check worker logs for the `tapping user audio input` / `tapping agent audio output` lines. If one is missing, that side will be silent on the merged recording. Confirm `api_key` is a WRITE-scope key and that `ROARK_OBSERVABILITY_ENABLED` is not set to a falsy value. *** ## Next Steps Send OpenTelemetry traces: they auto-link to the call via the room sid Test your LiveKit agents Set up evaluation criteria Use the dashboard-managed webhook integration instead # Okta SSO + SCIM Source: https://docs.roark.ai/documentation/integrations/okta Sign your team into Roark with Okta, and optionally push-provision and deprovision users via SCIM ## What this gets you Connecting Okta to Roark lets your team sign in with their Okta credentials. New users are auto-created in Roark on first sign-in (just-in-time provisioning), so you don't have to invite them individually. If you also enable SCIM, Okta becomes the source of truth for user lifecycle: assigning a user pushes them into Roark before their first login, profile updates flow through automatically, and deactivating the user in Okta disables their Roark account. *** ## How it works SAML and SCIM are two independent channels. You can run SAML-only (JIT provisioning) and add SCIM later, or run both from day one. Okta to Roark auth and provisioning flow: SAML through Cognito for sign-in, SCIM directly to Roark for user lifecycle * **SAML** authenticates the user. Okta posts the SAML response to Roark's Cognito User Pool, which establishes the session. * **SCIM** is a separate, outbound channel from Okta to Roark for user lifecycle events (create, update, deactivate). Roark verifies you own the email domain via a DNS TXT record before activating the SSO config, preventing accidental or malicious claims on a domain you don't control. *** ## Before you start * You are an **OWNER** of your Roark organization. * You have admin access to your Okta tenant. * You can publish a DNS TXT record at the apex of your email domain. * You know which email domain your team uses (e.g., `acmecorp.com`). One domain per Roark org. *** ## Step 1: Create the SAML app in Okta In your Roark dashboard, open **Org Settings → Single Sign-On**. The page shows two values you'll paste into Okta: the **ACS URL** and the **SP Entity ID**. Keep that tab open. In the Okta Admin Console: 1. Go to **Applications → Applications → Create App Integration**. 2. Choose **SAML 2.0** and click **Next**. 3. Give the app a name (e.g., "Roark") and continue. 4. On the SAML Settings screen, fill in: | Field | Value | | :-------------------------- | :----------------------------- | | Single sign-on URL (ACS) | (copy from Roark SSO settings) | | Audience URI (SP Entity ID) | (copy from Roark SSO settings) | | Name ID format | `EmailAddress` | | Application username | `Email` | 5. Add three **Attribute Statements**. Names are **case-sensitive**, type them exactly: | Name | Name format | Value | | :------------ | :---------- | :--------------- | | `email` | Basic | `user.email` | | `given_name` | Basic | `user.firstName` | | `family_name` | Basic | `user.lastName` | 6. Finish the wizard. On the resulting app's **Sign On** tab, find **Identity Provider metadata** and copy the URL. *** ## Step 2: Enable SAML in Roark Back in **Org Settings → Single Sign-On**: 1. Paste the IdP metadata URL. 2. Enter your email domain (e.g., `acmecorp.com`). 3. Click **Enable SSO**. Roark validates the metadata, registers the IdP with Cognito, and switches the settings page into the **pending verification** state with your ACS URL, Entity ID, and a DNS TXT record envelope displayed. Email addresses must match. If a user already has a Roark account under a different email than what Okta will send, you'll end up with duplicates. Reconcile email addresses in Okta before assigning users. *** ## Step 3: Verify your email domain After you enable SSO, Roark issues a one-time verification token and shows a DNS TXT record envelope on the settings page. Until you publish the record and Roark confirms it, the SSO config is inert: sign-ins for the domain fall through to the non-SSO path and any SCIM token you generate will not authenticate. Copy the values from the settings page into your DNS provider: | Field | Value | | :---- | :-------------------------------------------- | | Type | `TXT` | | Name | your email-domain apex (e.g., `acmecorp.com`) | | Value | `roark-domain-verification=` | 1. Add the record at the **apex** of your email domain, not a subdomain like `_roark.acmecorp.com`. It coexists with existing TXT records on the apex (SPF, DKIM, Google site verification, and so on). 2. Wait for propagation. This is typically a few minutes, but can take up to an hour depending on your provider's TTL. 3. Roark polls automatically in the background once the token is issued, so most admins won't need to click anything. Leave the settings page open for a few minutes after the DNS change propagates. There's a **Verify now** button for an explicit check. The token is valid for **7 days**. If it expires before you publish the record, click **Regenerate** to issue a fresh one. Regenerating invalidates any prior token, and if the row was already verified, clears that verified state as well (the new token becomes a re-attest). Until the domain is verified, the SSO config does not activate. Sign-ins for the domain are not routed through your IdP and SCIM bearer tokens will not authenticate. The settings page surfaces a **Pending verification** badge. If another Roark org has already verified the same email domain, your verify call returns `dns_record_mismatch`. Roark won't tell you which org owns it. [Contact support](/documentation/resources/support). *** ## Step 4: Assign users and test 1. In Okta, open the app's **Assignments** tab and assign at least one test user. 2. Have that user go to [roark.ai/login](https://roark.ai/login), enter their work email, and submit. 3. Roark detects SSO for the domain and redirects to Okta. After authenticating, the user lands in Roark, auto-created and joined to your org if they didn't already exist. If sign-in fails, jump to [Troubleshooting](#troubleshooting). *** ## Step 5 (optional): Turn on SCIM SCIM lets Okta push user lifecycle events into Roark, useful for larger teams and required if you want Okta to deactivate Roark accounts automatically. 1. In Roark, on **Org Settings → Single Sign-On → Okta**, click **Generate SCIM token**. Copy both the **base URL** and the **bearer token**. The token is shown once. 2) In Okta, open the same SAML app and go to the **Provisioning** tab. Click **Configure API Integration**. 3. Check **Enable API integration**, paste the SCIM base URL and bearer token, and click **Test API Credentials**. Save. 4. Under **Provisioning → To App**, enable: * **Create Users** * **Update User Attributes** * **Deactivate Users** **If you already have Roark users**, run **Provisioning → To App → Import Users from App** *before* assigning anyone in Okta. Okta calls Roark's SCIM endpoint, matches existing users by email, and records the link. Skipping this step still works (Roark matches by email on the first SCIM `POST` and returns the existing user), but importing first gives Okta a clean linked state and avoids confusing "User already exists" warnings in the Okta UI. 5. Assign users (or groups) to the app. Okta will create or link Roark accounts on assignment. *** ## Auto-join projects (optional) By default, new SSO and SCIM-provisioned users are added to your organization but no projects. To onboard them into a default set of projects automatically: 1. Go to **Org Settings → Single Sign-On → Auto-join projects**. 2. Pick the projects new users should land in. 3. Save. From then on, every user provisioned via JIT or SCIM is auto-added to those projects. Existing users are unaffected. *** ## What happens for existing Roark users Account linking only takes effect once the domain is verified. Until then, SCIM `POST /Users` is rejected (the token doesn't authenticate) and SAML sign-ins fall through to the non-SSO path. Roark links existing accounts (password or Google) to Okta by email: no duplicates, no data loss. The user keeps their user ID, history, and project memberships. * **JIT path:** linking happens on first SSO sign-in. Roark finds the user by email and merges Okta into their identity providers. * **SCIM path:** when Okta sends `POST /Users` for an existing user, Roark responds `200` (link) instead of `201` (create) and records the Okta `externalId`. The user does not need to sign in for this to take effect. If the email belongs to a different active Roark org, SCIM responds `409 (uniqueness)`. To resolve, remove the user from the other org and retry, or [contact support](/documentation/resources/support). *** ## Deprovisioning * **Unassign in Okta (SAML only):** the user can't start a new session. Existing sessions expire within \~1 hour. * **SCIM `PATCH active=false`:** the user is marked `INACTIVE`, their Cognito record is disabled, and their project memberships are removed. If you later re-activate them in Okta, their org membership and prior project memberships are restored. *** ## Disabling SSO From **Org Settings → Single Sign-On**, click **Disable SSO**. Roark removes the IdP from Cognito. Active sessions keep working until the token expires (within an hour). Users with a linked password or Google identity can still sign in that way; SSO-only users will be locked out until you re-enable. Disabling SSO does not clear domain verification. If you re-enable with the same email domain later, the prior verification still stands. If you re-enable with a different email domain, you'll need to verify the new one. *** ## Troubleshooting * **"Invalid SAML response received"**: Cognito couldn't fetch your IdP metadata. Confirm the metadata URL is publicly reachable and returns valid XML. * **User signs in but is treated as new**: the email in the Okta profile doesn't match the one on the existing Roark account. Update the Okta user's email to match, or contact support to relink. * **"Email domain is already claimed"**: another Roark org has already enabled SSO for this domain. Contact support. * **SSO check times out at login**: Roark couldn't look up the SSO provider for that domain. Check your network/firewall to roark.ai and confirm the email domain exactly matches what you configured. Domain verification returns one of the following results. The settings page surfaces these on every **Verify now** click; transient misses during background polling stay silent. * **`dns_record_not_found`**: no TXT record at the apex carries the `roark-domain-verification=` prefix. Confirm you added the record at the apex (not `_roark.acmecorp.com`) and allow time for DNS propagation, typically minutes, sometimes up to an hour depending on your provider's TTL. * **`dns_record_mismatch`**: a record with the `roark-domain-verification=` prefix exists, but its token doesn't match the one Roark issued. Two common causes: you regenerated the token after pasting the old value (paste the new one), or another organization is concurrently attempting to claim the same domain (only one will win: [contact support](/documentation/resources/support) if you believe this is your domain). * **`token_expired`**: the issued token is past its 7-day TTL. Click **Regenerate** and paste the fresh value. * **`dns_lookup_failed`**: DNS resolution itself errored. Usually transient; retry. If it persists, confirm the domain resolves at all (`dig TXT acmecorp.com`). * **`verified`**: success. The SSO config is now active end-to-end; SAML sign-ins and SCIM provisioning will route. *** ## Limitations * IdP-initiated SAML Single Logout is not supported. Logging out of Okta does not end Roark sessions; existing sessions expire on their own within \~1 hour. * One email domain per Roark org. If your team uses multiple domains, [reach out](/documentation/resources/support). * Wildcard or sub-domain claims are not supported. Verification and the email-domain match are exact at the apex only. * SCIM `GET /Users` only honors the `userName eq ""` filter. Okta only uses this form in practice; other filter expressions (`userName co`, `active eq true`, etc.) are ignored. *** ## Next steps Allow Roark's outbound IPs through your firewall Get help with Okta setup # Overview Source: https://docs.roark.ai/documentation/integrations/overview Every platform Roark connects to, and exactly which capabilities each one supports Roark connects to your agent platform in two directions: it **pulls production conversations in** for monitoring and scoring, and it **drives simulated conversations out** for testing. Which of those you get depends on the platform. This page is the map. Start with the capability matrix, then jump to the setup guide for your platform. *** ## Capability Matrix Supported out of the box  ·  Possible with extra setup  ·  Not supported | Platform | Import calls | Import chats | Sync agents | Telephony sims | WebRTC sims | Chat sims | Traces | | :--------------------------------------------------------------------------- | :-------------------: | :-------------------: | :---------------------------: | :---------------------------: | :-------------------: | :-------------------: | :---------------------------: | | [**VAPI**](/documentation/integrations/vapi) | | | | | | | | | [**Pipecat**](/documentation/integrations/pipecat) | | | | | | | | | [**Retell AI**](/documentation/integrations/retell) | | | | | | | | | [**ElevenLabs**](/documentation/integrations/elevenlabs) | | | | | | | | | [**Bland AI**](/documentation/integrations/bland) | | | | | | | | | [**Leaping**](/documentation/integrations/leaping) | | | | | | | | | [**LiveKit Cloud**](/documentation/integrations/livekit) | | | | | | | | | [**LiveKit (self-hosted)**](/documentation/integrations/livekit-self-hosted) | | | | | | | | | [**Google CES**](/documentation/integrations/google-ces) | | | | | | | | | [**Kore AI**](/documentation/integrations/kore) | | | | | | | | | [**Custom / API**](/documentation/integrations/custom-integrations) | | | | | | | | ### Notes on specific cells LiveKit is a real-time transport, not an agent framework, so it doesn't expose prompts or worker source code. Roark imports the **named agents** configured in your LiveKit project purely so calls can be attributed to the right agent. If you want prompts and tool definitions in Roark, use the [self-hosted SDK integration](/documentation/integrations/livekit-self-hosted), which captures the system prompt as a tracked prompt revision. Roark acts as an OpenTelemetry collector at `https://api.roark.ai/v1/traces`, so **any** platform can send traces over OTLP/HTTPS. Step-by-step guides exist for [VAPI](/documentation/observability/traces#vapi), [LiveKit](/documentation/observability/traces#livekit), and [custom integrations](/documentation/observability/traces#custom-integration). Platforms marked work through the custom-integration path: you instrument your own code and correlate spans to the call yourself. No provider currently pushes production **chat** transcripts into Roark. Text conversations reach Roark either from a [chat simulation](/documentation/simulation-testing/chat-simulations) or by posting them to the [`/v1/chat` API](/api-reference/introduction) yourself, which is why **Custom / API** is the only row with chat import. A WebRTC endpoint can be attached to any Roark agent whose underlying agent speaks **LiveKit** or **SmallWebRTC**. The matrix marks the platforms where that's the native, documented path. If your VAPI or ElevenLabs agent is fronted by one of those transports, you can wire up a WebRTC endpoint for it too. *** ## What Each Capability Means | Capability | What you get | | :----------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Import calls** | Production voice calls flow into Roark with audio, transcript, tool invocations, and metadata, then get scored automatically by your [metric collectors](/documentation/metrics/metric-collectors). | | **Import chats** | Production text conversations land as first-class [chat records](/documentation/simulation-testing/chat-simulations), evaluated the same way calls are. | | **Sync agents** | Agent configurations, system prompts, phone numbers, and tool definitions import automatically and stay up to date. Prompt changes are tracked as revisions. | | **Telephony sims** | Roark dials your agent over a real PSTN call, or answers when your agent dials Roark. See [inbound vs. outbound](/documentation/simulation-testing/inbound-vs-outbound). | | **WebRTC sims** | Roark joins your agent over [WebRTC](/documentation/simulation-testing/webrtc) instead of a phone line: no SIP, no telephony minutes, same transport as your web and mobile users. | | **Chat sims** | Roark drives persona-based [text conversations](/documentation/simulation-testing/chat-simulations) against your agent over its chat transport. | | **Traces** | OpenTelemetry spans attach to the matching call so you can debug latency, LLM calls, and tool failures on the [call detail page](/documentation/observability/traces). | *** ## Browse Integrations ### Voice agent platforms Sync assistants and stream calls in over webhooks Sync agents and analyze conversations Voice **and** chat, the only platform that does both Monitor Bland phone agents via pull-based sync Sync Leaping AI agents, prompts, and tools ### Real-time frameworks & self-hosted Dashboard-managed webhook integration, plus WebRTC endpoints Drop `observe_session` into your own Agents worker Drop the Roark observer into your Pipecat pipeline, self-hosted or on Pipecat Cloud ### Chat agent platforms Chat simulations against Customer Engagement Suite apps Chat simulations against Kore Agent Platform apps Any platform: send calls and chats with the API or SDKs Don't see your platform? [Custom integrations](/documentation/integrations/custom-integrations) cover every capability in the matrix. Send an audio file or a message transcript and Roark handles the rest. *** ## Simulation Transports Whether a simulation runs as a call or a chat is decided by the **endpoint type** on the agent target. You don't pick a modality separately. | Transport | Endpoint type | Modality | Best for | | :------------------- | :-------------------------------- | :------- | :--------------------------------------------------------------------------------------------------- | | **Telephony (PSTN)** | Phone | Voice | Any agent reachable at a phone number (the default for VAPI, Retell, ElevenLabs, Bland, and Leaping) | | **LiveKit** | LiveKit | Voice | Agents running in a LiveKit room, cloud or self-hosted | | **SmallWebRTC** | SmallWebRTC | Voice | Pipecat agents using peer-to-peer WebRTC with no media server | | **WebSocket** | WebSocket | Chat | Custom text agents you expose yourself | | **Provider chat** | ElevenLabs WS · Google CES · Kore | Chat | Auto-created when you connect the integration, nothing to wire up | ElevenLabs WS endpoints resolve to **chat**, not voice. To test an ElevenLabs agent over audio, use a Phone endpoint instead. *** ## How Integrations Work Pick your platform from the cards above and open its setup guide. Connect with an API key, OAuth, or (for self-hosted frameworks) a Roark API key dropped into your worker. Select which agents to track and how far back to backfill historical calls. Calls flow in and get scored automatically. Synced agents become targets you can point [run plans](/documentation/simulation-testing/run-plans) at. Data reaches Roark one of three ways, depending on the platform: **webhooks** (VAPI, Retell, LiveKit Cloud), **scheduled pull** (ElevenLabs, Bland, Leaping), or a **drop-in SDK** you add to your own code (LiveKit self-hosted, Pipecat, custom). *** ## One Integration, Many Agents A single integration syncs every agent in the connected account: ``` VAPI Integration ├── Customer Support Agent ├── Sales Qualification Agent ├── Appointment Booking Agent └── Survey Collection Agent ``` * **Single setup**: configure once, sync all agents * **Centralized management**: monitor every agent from one integration * **Unified analytics**: compare performance across agents and across platforms * **Simplified testing**: target any synced agent in a run plan *** ## Security & Privacy * **Encrypted connections**: TLS on every data transfer * **Secure storage**: credentials encrypted at rest and referenced by ID, never embedded in plans * **Access control**: role-based permissions for integration management * **Audit logging**: every integration activity is recorded * **Data isolation**: customer data separated by organization * **PII redaction**: strip sensitive data from transcripts and recordings with [PII redaction](/documentation/observability/pii-redaction) For network-level controls, see [IP whitelisting](/documentation/integrations/ip-whitelisting). *** ## Next Steps Every integration path starts here Score every imported call automatically Turn a synced agent into a test suite Push Roark events into your own systems # Pipecat Source: https://docs.roark.ai/documentation/integrations/pipecat Instrument a Pipecat voice agent with the roark_analytics[pipecat] observer, self-hosted or on Pipecat Cloud Live monitoring Voice simulations Chat simulations ## Overview The Pipecat integration monitors voice agents built on the open-source [Pipecat](https://github.com/pipecat-ai/pipecat) framework. Drop the [`roark_analytics[pipecat]`](https://pypi.org/project/roark-analytics/) observer into your pipeline and call lifecycle, transcripts, tool calls, and a recording are forwarded to Roark automatically. The same observer works whether you run Pipecat yourself or on **Pipecat Cloud**: the wiring is identical, so you pick your deployment when you connect the integration and instrument your pipeline once. The observer talks only to Roark's own API; it never changes how your agent runs. Deployment only matters for **simulations**. Monitoring (importing your production calls) is identical for self-hosted and Pipecat Cloud. If you only want monitoring, you can skip the simulation configuration below. *** ## Prerequisites Before setting up the integration, ensure you have: * A Pipecat voice agent (running on your own infrastructure or on Pipecat Cloud). * Python 3.10 or newer. * Access to your Roark project's **Agents** page to connect the integration and mint an API key. *** ## Setup Instructions ### Step 1: Connect the integration in Roark Go to **Agents**, click **Connect Agent**, choose **Existing Platform**, and pick **Pipecat**. Select the **Self-hosted** or **Pipecat Cloud** tab, give the integration a name, and (optionally) fill in the simulation configuration for that deployment (see [Simulations](#simulations) below). On connect, Roark mints a project API key **bound to this integration** and shows it once. Copy it now: it's what the observer authenticates with, and Roark can't show it again. ### Step 2: Install the observer ```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} pip install "roark_analytics[pipecat]" ``` ### Step 3: Wire the observer into your pipeline Add `RoarkObserver` to your Pipecat pipeline. Two rules: * Put `roark.audio_processor` **after** `transport.output()` so the observer captures the bot's post-TTS audio. * Pass `observers=[roark]` on `PipelineParams`. ```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}} from pipecat.pipeline.pipeline import Pipeline from pipecat.pipeline.runner import PipelineRunner from pipecat.pipeline.task import PipelineParams, PipelineTask from pipecat_roark import RoarkObserver SYSTEM_PROMPT = "You are a friendly voice assistant. Keep replies short." async def bot(runner_args): roark = RoarkObserver( api_key="", agent_id="pipecat-demo", agent_name="Pipecat Demo", agent_prompt=SYSTEM_PROMPT, runner_args=runner_args, ) # roark.audio_processor must sit AFTER transport.output() so the bot channel # captures post-TTS audio. The observer is wired via PipelineParams.observers. pipeline = Pipeline( [ transport.input(), stt, context_aggregator.user(), llm, tts, transport.output(), roark.audio_processor, context_aggregator.assistant(), ] ) task = PipelineTask( pipeline, params=PipelineParams( enable_metrics=True, enable_usage_metrics=True, observers=[roark], ), ) runner = PipelineRunner(handle_sigint=runner_args.handle_sigint) await runner.run(task) ``` The observer authenticates with the key from Step 1, sent as the `x-roark-api-key` header. Rather than hard-coding it, read it from the environment (`api_key=os.environ["ROARK_API_KEY"]`). On Pipecat Cloud, set `ROARK_API_KEY` as a deployment secret. `agent_id` and `agent_name` identify the agent in Roark. Agents are registered automatically on the first observed call, and `agent_prompt` is tracked as a prompt revision, so prompt changes show up in Roark over time. ### Step 4: Verify the connection Place a call to your agent. Within a few moments it appears in [Call History](/documentation/observability/live-monitoring) with its transcript, recording, and any tool calls, then gets scored by your active [collectors](/documentation/metrics/metric-collectors). *** ## How It Works The observer POSTs call lifecycle events (`call-started`, `call-ended`) to Roark's API, authenticated by your integration-bound key. Audio is streamed as chunks to Roark-issued presigned URLs and assembled into a recording when the call ends. Transcript and tool calls are sent in Pipecat's native format and mapped to Roark's model server-side. Because the key is bound to the integration, every call is attributed to the right integration and agent with no extra configuration. ## What Gets Synced * **Calls**: lifecycle, timing, and metadata for each conversation. * **Recording**: the mixed audio, assembled from the streamed chunks. * **Transcript**: turn-by-turn, from Pipecat's context messages. * **Tool calls**: function invocations captured during the call. * **Agent + prompt**: lazily registered from `agent_id` / `agent_name`, with `agent_prompt` tracked as a revision. *** ## Simulations To run [simulations](/documentation/simulation-testing/overview) against a Pipecat agent, Roark joins it over [WebRTC](/documentation/simulation-testing/webrtc). Add the simulation configuration when you connect the integration (or edit it later). This is the one part that differs by deployment. Roark reaches your agent through its **SmallWebRTC** signaling endpoint. | Field | Description | | :------------------ | :------------------------------------------------------------------------------------------------------ | | **Signaling URL** | Your SmallWebRTC offer endpoint. Roark POSTs an SDP offer here (`http(s)` or `ws(s)`). | | **Auth token** | Optional bearer token for the signaling server. Stored encrypted. | | **Append agent ID** | Append each synced agent's id as a URL path segment, so one host can route to many bots. On by default. | Roark creates a WebRTC simulation endpoint for each synced agent automatically. Roark starts a session through **Pipecat Cloud**, which runs your agent over Daily. | Field | Description | | :------------------ | :--------------------------------------------------------------------------------- | | **Public API key** | Your Pipecat Cloud **public** API key, used to start sessions. Stored encrypted. | | **Start body** | Optional JSON forwarded as the agent `body` on the Pipecat Cloud `/start` request. | | **Room properties** | Optional JSON forwarded as the Daily `dailyRoomProperties`. | The Pipecat Cloud start URL is derived from each synced agent's name (its Pipecat Cloud deployment name), so there's nothing else to configure per agent. *** ## Next Steps Test your Pipecat agent with synthetic callers over WebRTC Score every incoming call automatically # Retell AI Source: https://docs.roark.ai/documentation/integrations/retell Sync Retell agents and analyze voice AI conversations Live monitoring Voice simulations Chat simulations ## Overview The Retell AI integration enables comprehensive monitoring of your Retell voice agents. Sync historical calls, monitor live conversations, and automatically evaluate agent performance using Roark's analytics platform. *** ## Prerequisites Before setting up the integration, ensure you have: * A Retell AI account with active agents * Retell API key (found in your Retell dashboard) * At least one configured agent in Retell *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **Retell AI** 3. Enter your configuration: | Field | Description | | :------------------- | :--------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your Retell API key | The system will automatically validate your API key and fetch available agents. ### Step 2: Configure Historical Sync Choose how far back to import existing calls: * **Default**: Last 90 days of call history * **Custom**: Select a specific date range * **Skip**: Only sync new calls going forward Historical sync runs once during initial setup. Future calls are synced automatically via webhooks. ### Step 3: Select Agents Choose which Retell agents to monitor: * **All Agents** - Sync calls from every agent in your account * **Selected Agents** - Choose specific agents to monitor The agent selector shows: * Agent name * Agent ID * Last modified date ### Step 4: Configure Metric Collectors Calls synced from Retell are scored by your [metric collectors](/documentation/metrics/metric-collectors). Set them up globally under **Metrics → Collectors**, or scope collectors to a specific agent from that agent's **Collectors** section. ### Step 5: Activate Integration Review your settings and click **Create Integration** to begin syncing. *** ## What Gets Synced Retell integrations sync comprehensive data: * **Calls** - Complete conversation data and metadata * **Agents** - Agent configurations and phone numbers * **Prompts** - System prompts and instructions * **Transcripts** - Full conversation transcripts with speaker labels * **Tool Calls** - Function/tool invocations during conversations * **Call Metadata** - Duration, status, participant information *** ## Webhook Configuration Roark automatically configures webhooks with Retell. The integration handles: | Event | Description | Roark Action | | :------------- | :------------- | :----------------------------------- | | `call_started` | Call initiated | Create call record, begin monitoring | | `call_ended` | Call completed | Process transcript, run evaluations | The webhook URL is automatically registered with Retell using your API key. *** ## Sync Management ### Initial Sync During setup, Roark imports historical calls based on your selected date range: * Processes calls in batches * Shows sync progress in dashboard * Handles large volumes efficiently ### Ongoing Sync After initial setup: * New calls sync automatically via webhooks * Real-time processing of live calls * Automatic retry on temporary failures ### Resume Sync If sync is interrupted: 1. Go to integration settings 2. Click **Resume Sync** 3. Choose to continue from last position or restart *** ## Monitoring Integration Health Track your integration status: * **Syncing** 🔄 - Actively importing historical calls * **Active** 🟢 - Receiving new calls via webhook * **Paused** ⏸️ - Sync temporarily halted * **Error** ⚠️ - Configuration or connection issues View sync statistics: * Total calls synced * Last sync timestamp * Sync progress percentage * Recent webhook deliveries *** ## Agent Management Synced agents appear in: * **Simulation agent selection** - Use for testing * **Agent performance reports** - Track metrics per agent * **Comparison dashboards** - Analyze across agents Agent data updates automatically when modified in Retell. *** ## Next Steps Test synced agents Analyze call performance Define metrics and pass/fail thresholds Explore other integrations # VAPI Source: https://docs.roark.ai/documentation/integrations/vapi Sync VAPI assistants and analyze voice AI conversations Live monitoring Voice simulations Chat simulations ## Overview The VAPI integration enables comprehensive monitoring of your VAPI voice assistants. Sync historical calls, monitor live conversations, and automatically evaluate assistant performance using Roark's analytics platform. *** ## Prerequisites Before setting up the integration, ensure you have: * A VAPI account with active assistants * VAPI API key (found in your VAPI dashboard) * At least one configured assistant in VAPI *** ## Setup Instructions ### Step 1: Create Integration 1. Navigate to **Agents** in your Roark dashboard 2. Click **Connect Agent** in the top right, choose **Existing Platform**, and select **VAPI** 3. Enter your configuration: | Field | Description | | :------------------- | :--------------------------------- | | **Integration Name** | Friendly name for this integration | | **API Key** | Your VAPI API key | The system will automatically validate your API key and fetch available assistants. ### Step 2: Configure Historical Sync Choose how far back to import existing calls: * **Default**: Last 90 days of call history * **Custom**: Select a specific date range * **Skip**: Only sync new calls going forward Historical sync runs once during initial setup. Future calls are synced automatically via webhooks. ### Step 3: Select Assistants Choose which VAPI assistants to monitor: * **All Assistants** - Sync calls from every assistant in your account * **Selected Assistants** - Choose specific assistants to monitor The assistant selector shows: * Assistant name * Assistant ID * Last modified date ### Step 4: Configure Metric Collectors Calls synced from VAPI are scored by your [metric collectors](/documentation/metrics/metric-collectors). Set them up globally under **Metrics → Collectors**, or scope collectors to a specific agent from that agent's **Collectors** section. ### Step 5: Activate Integration Review your settings and click **Create Integration** to begin syncing. *** ## What Gets Synced VAPI integrations sync comprehensive data: * **Calls** - Complete conversation data and metadata * **Assistants** - Assistant configurations and phone numbers * **Prompts** - System prompts and instructions * **Transcripts** - Full conversation transcripts with speaker labels * **Tool Calls** - Function/tool invocations during conversations * **Call Metadata** - Duration, status, participant information *** ## Webhook Configuration Roark automatically configures webhooks with VAPI. The integration handles: | Event | Description | Roark Action | | :------------- | :------------- | :----------------------------------- | | `call_started` | Call initiated | Create call record, begin monitoring | | `call_ended` | Call completed | Process transcript, run evaluations | The webhook URL is automatically registered with VAPI using your API key. *** ## Sync Management ### Initial Sync During setup, Roark imports historical calls based on your selected date range: * Processes calls in batches * Shows sync progress in dashboard * Handles large volumes efficiently ### Ongoing Sync After initial setup: * New calls sync automatically via webhooks * Real-time processing of live calls * Automatic retry on temporary failures ### Resume Sync If sync is interrupted: 1. Go to integration settings 2. Click **Resume Sync** 3. Choose to continue from last position or restart *** ## Monitoring Integration Health Track your integration status: * **Syncing** 🔄 - Actively importing historical calls * **Active** 🟢 - Receiving new calls via webhook * **Paused** ⏸️ - Sync temporarily halted * **Error** ⚠️ - Configuration or connection issues View sync statistics: * Total calls synced * Last sync timestamp * Sync progress percentage * Recent webhook deliveries *** ## Assistant Management Synced assistants appear in: * **Simulation assistant selection** - Use for testing * **Assistant performance reports** - Track metrics per assistant * **Comparison dashboards** - Analyze across assistants Assistant data updates automatically when modified in VAPI. *** ## Next Steps Traces sync automatically. See how to enable them Test synced assistants Analyze call performance Explore other integrations # Webhooks Source: https://docs.roark.ai/documentation/integrations/webhooks Receive real-time notifications when call analysis completes or fails ## Overview Webhooks allow you to receive real-time HTTP notifications when events occur in Roark. Instead of polling for updates, Roark will automatically send event data to your specified endpoint whenever a call analysis completes or fails.