Get job evaluation runs

GET

evaluation

job

{jobId}

runs

JavaScript

import Roark from '@roarkanalytics/sdk';

const client = new Roark({
  bearerToken: process.env['ROARK_API_BEARER_TOKEN'], // This is the default and can be omitted
});

const response = await client.evaluation.getJobRuns('jobId');

console.log(response.data);

{
  "data": {
    "pagination": {
      "total": 123,
      "hasMore": true,
      "nextCursor": "<string>"
    },
    "data": [
      {
        "evaluator": {
          "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
          "name": "<string>",
          "weight": 123
        },
        "status": "PENDING",
        "metrics": [
          {
            "name": "<string>",
            "valueType": "<string>",
            "numericValue": 123,
            "booleanValue": true,
            "textValue": "<string>",
            "confidence": 0.5,
            "reasoning": "<string>",
            "role": "<string>",
            "createdAt": "<string>"
          }
        ],
        "evidence": [
          {
            "snippetText": "<string>",
            "commentText": "<string>",
            "isPositive": true,
            "createdAt": "<string>"
          }
        ],
        "blockRuns": [
          {
            "blockRunId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
            "blockDefinitionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
            "blockName": "<string>",
            "status": "PENDING",
            "result": "PASSED",
            "score": 0.5,
            "reason": "<string>",
            "createdAt": "<string>"
          }
        ],
        "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "score": 0.5,
        "result": "SUCCESS",
        "summary": "<string>",
        "startedAt": "<string>",
        "completedAt": "<string>"
      }
    ]
  }
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

jobId

string

required

Query Parameters

nextCursor

string

Cursor for the next page of items

limit

string

Number of items to return per page

Required string length: 1 - 50

Example:

"10"

Response

Successfully retrieved evaluation job runs

data

object

required

Evaluation job runs response payload

Show child attributes

data.pagination

object

required

Pagination information

Show child attributes

data.pagination.total

number

required

Total number of items

data.pagination.hasMore

boolean

required

Whether there are more items to fetch

data.pagination.nextCursor

string | null

required

Cursor for the next page of items

data.data

object[] | null

required

Evaluator runs of the evaluation job

Show child attributes

data.data.evaluator

object

required

Show child attributes

data.data.evaluator.id

string<uuid>

required

ID of the evaluator

data.data.evaluator.name

string

required

Name of the evaluator

data.data.evaluator.weight

number

Weight of the evaluator

data.data.status

enum<string>

required

Status of the evaluator run

Available options:

PENDING,

IN_PROGRESS,

COMPLETED,

FAILED

data.data.metrics

object[]

required

Show child attributes

data.data.metrics.name

string

required

Name of the metric

data.data.metrics.valueType

string

required

Value type of the metric

data.data.metrics.numericValue

number | null

required

Numeric value of the metric

data.data.metrics.booleanValue

boolean | null

required

Boolean value of the metric

data.data.metrics.textValue

string | null

required

Text value of the metric

data.data.metrics.confidence

number | null

required

Confidence level of the metric (0-1)

Required range: 0 <= x <= 1

data.data.metrics.reasoning

string | null

required

Reasoning of the metric

data.data.metrics.role

string

required

Role of the metric

data.data.metrics.createdAt

string

required

Created at of the metric

data.data.evidence

object[]

required

Show child attributes

data.data.evidence.snippetText

string

required

Snippet text of the evidence

data.data.evidence.commentText

string | null

required

Comment text of the evidence

data.data.evidence.isPositive

boolean

required

Is positive of the evidence

data.data.evidence.createdAt

string

required

Created at of the evidence

data.data.blockRuns

object[]

required

All block runs for this evaluator, including skipped ones

Show child attributes

data.data.blockRuns.blockRunId

string<uuid>

required

ID of the block run instance

data.data.blockRuns.blockDefinitionId

string<uuid>

required

ID of the block definition

data.data.blockRuns.blockName

string

required

Name of the evaluation block

data.data.blockRuns.status

enum<string>

required

Status of the block run

Available options:

PENDING,

RUNNING,

COMPLETED,

FAILED

data.data.blockRuns.result

enum<string> | null

required

Result of the block run

Available options:

PASSED,

FAILED,

SKIPPED

data.data.blockRuns.score

number | null

required

Score of the block run (0-1)

Required range: 0 <= x <= 1

data.data.blockRuns.reason

string | null

required

Reason for the outcome (pass/fail explanation or skip reason)

data.data.blockRuns.createdAt

string

required

When the block run was created

data.data.id

string<uuid>

ID of the evaluator run

data.data.score

number | null

Score of the evaluation run (0-1)

Required range: 0 <= x <= 1

data.data.result

enum<string> | null

Result of the evaluator run based on score threshold (IRRELEVANT is mapped to SKIPPED)

Available options:

SUCCESS,

FAILURE,

SKIPPED

data.data.summary

string | null

Summary of the evaluation run

data.data.startedAt

When the evaluator run started

data.data.completedAt

When the evaluator run completed

Create call evaluation jobCreate a evaluation job for a single call or dataset of calls

⌘I

JavaScript

import Roark from '@roarkanalytics/sdk';

const client = new Roark({
  bearerToken: process.env['ROARK_API_BEARER_TOKEN'], // This is the default and can be omitted
});

const response = await client.evaluation.getJobRuns('jobId');

console.log(response.data);

{
  "data": {
    "pagination": {
      "total": 123,
      "hasMore": true,
      "nextCursor": "<string>"
    },
    "data": [
      {
        "evaluator": {
          "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
          "name": "<string>",
          "weight": 123
        },
        "status": "PENDING",
        "metrics": [
          {
            "name": "<string>",
            "valueType": "<string>",
            "numericValue": 123,
            "booleanValue": true,
            "textValue": "<string>",
            "confidence": 0.5,
            "reasoning": "<string>",
            "role": "<string>",
            "createdAt": "<string>"
          }
        ],
        "evidence": [
          {
            "snippetText": "<string>",
            "commentText": "<string>",
            "isPositive": true,
            "createdAt": "<string>"
          }
        ],
        "blockRuns": [
          {
            "blockRunId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
            "blockDefinitionId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
            "blockName": "<string>",
            "status": "PENDING",
            "result": "PASSED",
            "score": 0.5,
            "reason": "<string>",
            "createdAt": "<string>"
          }
        ],
        "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
        "score": 0.5,
        "result": "SUCCESS",
        "summary": "<string>",
        "startedAt": "<string>",
        "completedAt": "<string>"
      }
    ]
  }
}

API Reference

Health

Integrations

Call

Chat

Evaluation

Simulation

Persona

Metric Definitions

Get job evaluation runs

Authorizations

Path Parameters

Query Parameters

Response