Collection Jobs

Overview

Collection jobs are the programmatic way to run metrics on demand for one or many calls via the API. While collectors automate collection for incoming calls, collection jobs let you trigger metric processing against existing calls — whether that’s backfilling a new metric across historical data or re-running after a definition change.

Want to run metrics across calls from the UI? Use Studio in Evaluate mode (“Run a battery of metrics across a set of calls”) instead. Collection jobs are designed for programmatic, at-scale use via the SDK.

When to Use Collection Jobs

Backfill metrics — Run newly created metrics on historical calls
Re-run metrics — Reprocess calls after updating a metric definition
On-demand analysis — Collect metrics for a specific set of calls uploaded via API
Scale up after testing — You’ve validated a metric in Studio, now run it across many calls programmatically

Creating a Collection Job

Provide an array of call IDs and the metrics you want to collect:

const job = await client.metricCollectionJob.create({
  callIds: ['call-uuid-1', 'call-uuid-2', 'call-uuid-3'],
  metrics: [
    { id: 'metric-definition-uuid-1' },
    { id: 'metric-definition-uuid-2' },
  ],
})

// job.data
{
  id: 'job-uuid',
  status: 'PENDING',
  triggeredBy: 'USER_API',
  totalItems: 6,
  completedItems: 0,
  failedItems: 0,
  startedAt: null,
  completedAt: null,
  errorMessage: null,
  createdAt: '2025-01-15T10:30:00Z',
  updatedAt: '2025-01-15T10:30:00Z',
}

Parameters:

Field	Type	Required	Description
`callIds`	string[]	Yes	Array of call UUIDs to process (minimum 1)
`metrics`	array	Yes	Metric definitions to collect (minimum 1)

The totalItems count equals callIds.length * metrics.length — each call-metric combination is a separate item.

Job Lifecycle

Collection jobs progress through the following statuses:

PENDING → PROCESSING → COMPLETED / FAILED / CANCELED

Status	Description
`PENDING`	Job created, waiting to start processing
`PROCESSING`	Actively collecting metrics from calls
`COMPLETED`	All items processed successfully
`FAILED`	Job encountered errors (check `errorMessage`)
`CANCELED`	Job was canceled before completion

Progress Tracking

Monitor progress using these fields:

Field	Description
`totalItems`	Total number of call-metric combinations to process
`completedItems`	Number of items successfully processed
`failedItems`	Number of items that failed
`startedAt`	When processing began
`completedAt`	When processing finished

Monitoring Jobs

List Collection Jobs

const jobs = await client.metricCollectionJob.list()

// With filters
const completedJobs = await client.metricCollectionJob.list({
  status: 'COMPLETED',
  limit: 50,
})

Parameter	Type	Description
`limit`	number	Max results (1-50, default: 20)
`after`	string	Cursor for pagination
`status`	string	Filter by `PENDING`, `PROCESSING`, `COMPLETED`, `FAILED`, or `CANCELED`

Get a Collection Job

const job = await client.metricCollectionJob.getByID('job-id')

Returns the full job object with current progress.

Example: Studio to Production

A typical end-to-end workflow — test a metric in Studio, then run it across calls via the SDK:

Test in Studio

Use Studio to test your metric prompt against a sample call. Iterate until you’re happy with the output.

Create the Metric Definition

Once validated, create the metric definition via the SDK:

const metric = await client.metric.createDefinition({
  name: 'Identity Verified',
  outputType: 'BOOLEAN',
  analysisPackageId: 'your-package-id',
  llmPrompt: 'Did the agent successfully verify the caller identity?',
  booleanTrueLabel: 'Verified',
  booleanFalseLabel: 'Not Verified',
})

Run on Existing Calls

Create a collection job targeting the calls you want to analyze:

const job = await client.metricCollectionJob.create({
  callIds: ['call-id-1', 'call-id-2', 'call-id-3'],
  metrics: [{ id: metric.data.id }],
})

Poll for Completion

let status = 'PENDING'
while (status !== 'COMPLETED' && status !== 'FAILED') {
  const result = await client.metricCollectionJob.getByID(job.data.id)
  status = result.data.status
  console.log(`Progress: ${result.data.completedItems}/${result.data.totalItems}`)
}

Fetch Results

const metrics = await client.call.listMetrics('call-id-1')
console.log(metrics.data)

Once you’ve validated the metric works well across calls, consider creating a collector to automatically collect it on future calls.

Getting Started

Observability

Metrics

Simulations

Config as Code

Recipes

Integrations

SDKs & Libraries

Resources

Overview

When to Use Collection Jobs

Creating a Collection Job

Job Lifecycle

Progress Tracking

Monitoring Jobs

List Collection Jobs

Get a Collection Job

Example: Studio to Production

What’s Next

Studio

Collectors

​Overview

​When to Use Collection Jobs

​Creating a Collection Job

​Job Lifecycle

​Progress Tracking

​Monitoring Jobs

​List Collection Jobs

​Get a Collection Job

​Example: Studio to Production

​What’s Next

Studio

Collectors

Overview

When to Use Collection Jobs

Creating a Collection Job

Job Lifecycle

Progress Tracking

Monitoring Jobs

List Collection Jobs

Get a Collection Job

Example: Studio to Production

What’s Next