For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to page URLs.
Skip to content

API

Management API endpoints for AI Testing and Evaluation.

The Test API provides endpoints for creating, managing, and executing AI tests, as well as querying and comparing test runs.

Test Endpoints

Method
Endpoint
Description

GET

List all tests

GET

Get a test by ID or alias

POST

Create a new test

PUT

Update an existing test

DELETE

Delete a test

POST

Execute a single test

POST

Execute multiple tests

POST

Execute tests by tags

Test Run Endpoints

Method
Endpoint
Description

GET

List test runs (filtered)

GET

/test-runs/{id}

Get a test run by ID

GET

/test-runs/{id}/transcript

Get the transcript for a run

GET

/test-runs/latest/{testId}

Get the most recent run for a test

GET

/test-runs/executions/{executionId}

Get execution summary with per-variation metrics

DELETE

/test-runs/{id}

Delete a test run

POST

/test-runs/baseline/{testId}/{testRunId}

Set a run as the test baseline

POST

Compare two test runs

POST

Compare two variation groups

Base URL

Test Object

Properties

Property
Type
Description

id

guid

Unique identifier

alias

string

Unique alias for code references

name

string

Display name

description

string

Optional description

testFeatureId

string

Test feature harness (prompt, agent)

testTargetId

guid

ID of the target entity

profileId

guid

Default AI profile (optional)

contextIds

guid[]

Default AI contexts

testFeatureConfig

object

Feature-specific configuration (JSON)

graders

array

Success criteria

variations

array

A/B testing overrides

runCount

int

Number of runs per execution

tags

string[]

Organization tags

baselineRunId

guid

Baseline run for regression detection

dateCreated

date

When the test was created (UTC)

dateModified

date

When the test was last modified (UTC)

version

int

Current version number

Last updated

Was this helpful?