Skip to main content

Overview

Tool invocations allow you to capture and analyze the tools used during calls. This documentation covers how to submit tool calls, parameter options, result formats, and how Roark tracks versioning for tools.

Submitting Tool Calls

Tool calls can be submitted as part of your post-call-analysis by including a toolInvocations array in your request payload when you create the call. If your tool data is not ready at call-creation time, you can also attach it afterward. Each tool invocation requires:
  • name: The name of the tool being invoked
  • description: A brief description of what the tool does
  • startOffsetMs: The start time offset in milliseconds with respect to the start of the call
  • endOffsetMs: The end time offset in milliseconds with respect to the start of the call
  • parameters: The parameters passed to the tool
  • result: The result of the tool invocation

Attaching Tool Calls After a Call

Sometimes your tool data is not ready when the call is created, for example when a Roark simulation has finished but your backend needs a moment to assemble the tool calls, or when you submit a call before its tools are available. In those cases you can attach tool invocations to an existing call:
The body takes the same toolInvocations array as call creation:
The response reports how many invocations were added and how many were skipped.
Attaching is idempotent. Re-sending an invocation that is already on the call (same tool name and timing) is skipped rather than duplicated, so a retried request converges instead of double-counting.
To score the tool metrics over the newly attached tools in the same request, pass a metrics array of metric definition IDs. This starts a metric collection job (billed, and requires the metric:create permission). Omit it to attach tools only. If scoring can’t be started (for example, insufficient credits), the tools are still attached and the response returns a metricCollectionError you can act on.

Tracking Input Parameters

Parameters can be submitted in two different formats:

Simple Key-Value Pairs

The most basic format is a simple key-value pair. Roark will automatically infer the type of the parameter, and rely on the name of the parameter to understand the context.

Detailed Key-Value Pairs

To ensure that the usage of the tool is clear and the types are correctly inferred, you can manually provide the description and type for each parameter. Type can be one of: string, number, boolean.

Tracking Results

The result returned by the tool can be a string or a JSON object.

String Results

For simple responses, you can pass in a string:

JSON Object Results

For structured data, you can pass a JSON object:

Tool Execution Duration

Roark calculates execution duration automatically using the difference between startOffsetMs and endOffsetMs:
  • startOffsetMs: Time offset (in milliseconds) from the beginning of the call when the tool execution started
  • endOffsetMs: Time offset (in milliseconds) from the beginning of the call when the tool execution ended

Tool Versioning

The system automatically tracks changes to your tool calls and creates new versions when:
  1. A tool with the same name receives a different schema for input parameters
  2. A tool with the same name is called with a different description

Example

If you previously called a tool:
And later change it to:
The system will automatically create a new version to track these changes.