Skip to main content

Overview

Collection jobs are the programmatic way to run metrics on demand for one or many calls via the API. While collectors automate collection for incoming calls, collection jobs let you trigger metric processing against existing calls, whether that’s backfilling a new metric across historical data or re-running after a definition change.
Want to run metrics across calls from the UI? Use Studio in Evaluate mode (“Run a battery of metrics across a set of calls”) instead. Collection jobs are designed for programmatic, at-scale use via the SDK.

When to Use Collection Jobs

  • Backfill metrics: Run newly created metrics on historical calls
  • Re-run metrics: Reprocess calls after updating a metric definition
  • On-demand analysis: Collect metrics for a specific set of calls uploaded via API
  • Scale up after testing: You’ve validated a metric in Studio, now run it across many calls programmatically

Creating a Collection Job

Provide an array of call IDs and the metrics you want to collect:
Parameters:
The totalItems count equals callIds.length * metrics.length: each call-metric combination is a separate item.

Re-scoring with collector settings

Without policyId, each metric runs with its defaults. Metrics in a collector can carry per-collector settings, such as which properties Property Verification checks. To re-score calls exactly as a collector scores them, pass its id as policyId:
  • Each metric runs as the collector runs it, with its settings. metrics defaults to every metric the collector collects; if you list metrics, each must be in the collector.
  • The collector’s conditions are not applied: the calls you list are scored as given.
  • The job and the values it writes carry the collector’s id in policyIds, as a live collection does.
  • One request covers at most 5,000 call-metric items, so with a collector of 12 metrics that is about 400 calls per request.
Collector ids are returned by client.metricPolicy.list() (GET /v1/metric/policies).

Job Lifecycle

Collection jobs progress through the following statuses:

Progress Tracking

Monitor progress using these fields:

Monitoring Jobs

List Collection Jobs

Get a Collection Job

Returns the full job object with current progress.

Example: Studio to Production

A typical end-to-end workflow (test a metric in Studio, then run it across calls via the SDK):
1

Test in Studio

Use Studio to test your metric prompt against a sample call. Iterate until you’re happy with the output.
2

Create the Metric Definition

Once validated, create the metric definition via the SDK:
3

Run on Existing Calls

Create a collection job targeting the calls you want to analyze:
4

Poll for Completion

5

Fetch Results

Once you’ve validated the metric works well across calls, consider creating a collector to automatically collect it on future calls.

What’s Next

Studio

Test metrics interactively before running at scale

Collectors

Automate metric collection for incoming calls