For the complete documentation index, see llms.txt. This page is also available as Markdown.

Evaluations and Test Sets

The evaluation and test set endpoints correspond to "Test Sets" and "Automated Testing" in the AgentOps back office, letting you batch-test Agents from your own systems. The full flow is:

  1. Create a test set: Use POST /api/v1/dataset/ to create an empty test set, or use POST /api/v1/dataset/upload/ to upload a CSV/Excel file and create one in a single step.

  2. Add test cases: For single-turn Q&A, use /api/v1/test-case/ (create one by one, upload a file, or generate from a knowledge base file); for multi-turn conversation scenarios, use /api/v1/multi-turn-test-case/. Generating from a knowledge base file runs in the background and returns 202 with a jobId; track it with Retrieve test case generation job progress.

  3. Run an evaluation: Use POST /api/v1/evaluations/evaluate/ (single-turn) or POST /api/v1/evaluations/multi-turn-evaluate/ (multi-turn) to specify the test set and Agent. The response is 202 with an evaluationId.

  4. Read results and export: Poll GET /api/v1/evaluations/{id}/ until status becomes completed, partial_completed, or failed. Per-question results for single-turn evaluations are in details, and per-conversation results for multi-turn evaluations are in multi-turn-results. A completed single-turn evaluation can be exported as an Excel report with export, and the download URL is available from export-status.

Required permissions: The API Key calls the API as the member who created it. Listing and all write operations on /dataset/, /test-case/, and /multi-turn-test-case/ require a role with the "Test Set permission"; listing and all write operations on /evaluations/ require the "Automated Testing permission" (the AgentOps permissions in Role Permissions; owners always have them). Insufficient permissions return 403. Members other than the owner can only see and evaluate Agents that their role is allowed to use.

Credit: Generating test cases from knowledge base files deducts the pricing item agentops_test_gen based on the number of questions actually generated. Running an evaluation deducts agentops_test_run and agentops_deepeval when it finishes, based on the number of completed test cases, and the evaluation model calls used for scoring are billed separately according to that model's usage. Before a generation or evaluation request is submitted, the system checks the organization's Credit and quota, and returns 402 if they are insufficient. Manual creation, file upload, and reading results do not deduct Credit; for unit prices, see List pricing items.

For features and back-office operations, see the user manual pages Test Set Management and Automated Evaluation and Agent Monitoring.

List Test Sets

GET /api/v1/dataset/

Parameters

Parameter
Required
Type
Description

isActive

❌

boolean

List only active (true) or inactive (false) datasets

ordering

❌

string

Sort field: created_at or updated_at. Prefix with - for newest first (e.g. -updated_at). If not specified, sorted by creation time, newest first

page

❌

integer

Page number, starting from 1. Returns 404 if beyond the last page

pageSize

❌

integer

Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)

query

❌

string

Keyword; matches the dataset name, description, or ID (partial match)

search

❌

string

Keyword; matches the dataset name or description (partial, case-insensitive match; when multiple keywords are separated by spaces, all must match)

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values


Create an empty test set

POST /api/v1/dataset/

Test set names must be unique within an organization; a duplicate name returns 400.

Request Content

Request Parameters

Field
Type
Required
Description

name

string

Yes

description

string

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status code: 201

Example Response Structure

Example Response Values


Upload CSV or Excel to create a test set

POST /api/v1/dataset/upload/

Upload as multipart/form-data: file and name are required, and description is optional. The file format is the same as Upload CSV or Excel to add test cases; test set names must be unique within an organization.

Code Examples

Response Content

Status code: 201

Example Response Structure

Example Response Values


Generate a test set from a knowledge base file

POST /api/v1/dataset/generate-from-file/

Creates a new test set from a single knowledge base file (knowledgeBaseFileId) and generates single-turn test cases; numTestCases ranges from 1 to 50 (default 10). Generation runs in the background, and the actual response is 202 with a taskId; it does not include the test set ID, and there is no progress to query. Once generation finishes, the new test set appears in the list. If you need to track progress, first create an empty test set and then use Generate single-turn test cases from a knowledge base file. Credit: The organization's Credit is checked before submission, and 402 is returned if it is insufficient; agentops_test_gen is deducted based on the number of questions actually generated.

Request Content

Request Parameters

Field
Type
Required
Description

name

string

Yes

description

string

No

chatbotFileId

string (uuid)

No

knowledgeBaseFileId

string (uuid)

No

numTestCases

integer

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status code: 202

Example Response Structure

Example Response Values


List knowledge base files available for test case generation

GET /api/v1/dataset/knowledge-base-files/

Grouped by knowledge base, lists only fully processed files of type txt, docx, pdf, md, markdown, or mdx. pageSize is the maximum number of files returned per knowledge base (default 5, maximum 100); search filters by filename.

Parameters

Parameter name
Required
Type
Description

pageSize

❌

integer

Maximum number of files returned per knowledge base (default 5, maximum 100)

search

❌

string

Filter by filename (case-insensitive partial match)

Code examples

Response body

Status code: 200

Example response schema

Example response values


Retrieve a specific test set

GET /api/v1/dataset/{id}/

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Dataset ID

Code Examples

Response Content

Status code: 200

Example Response Structure

Example Response Values


Retrieve a test set with its single-turn test cases

GET /api/v1/dataset/{id}/details/

In addition to the test set fields, the response includes a testCases array that contains only single-turn test cases; for multi-turn test cases, use GET /api/v1/multi-turn-test-case/?datasetId={id}.

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Dataset ID

Code Examples

Response Content

Status code: 200

Example Response Structure

Example Response Values


Update a test set

PUT /api/v1/dataset/{id}/

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Dataset ID

Request Content

Request Parameters

Field
Type
Required
Description

name

string

Yes

description

string

No

isActive

boolean

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values


Partially update a test set

PATCH /api/v1/dataset/{id}/

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Dataset ID

Request Content

Request Parameters

Field
Type
Required
Description

name

string

No

description

string

No

isActive

boolean

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values


Delete a test set

DELETE /api/v1/dataset/{id}/

This cannot be undone. Deleting a test set also deletes all test cases in it, as well as the evaluations and results that were run with this test set.

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Dataset ID

Code Examples

Response Content

Status code
Description

204

No response body


List Single-Turn Test Cases

GET /api/v1/test-case/

Use datasetId to filter by a single test set. You can also filter by status and creationMethod (manual, file, generate).

Parameters

Parameter
Required
Type
Description

creationMethod

❌

string

List only test cases created by this method. Allowed values: `manual`: Manual ; `file`: File import ; `generate`: Generated;

datasetId

❌

string

List only test cases in this dataset

ordering

❌

string

Sort field: created_at or updated_at. Prefix with - for newest first (e.g. -updated_at). If not specified, sorted by creation time, newest first

page

❌

integer

Page number, starting from 1. Returns 404 if beyond the last page

pageSize

❌

integer

Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)

query

❌

string

Keyword; matches the question, ground truth, or ID (partial match)

search

❌

string

Keyword; matches the question or ground truth (partial, case-insensitive match; when multiple keywords are separated by spaces, all must match)

status

❌

string

List only test cases with this status. Allowed values: `active`: Active ; `inactive`: Inactive ; `pending`: Pending Review;

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Create a Single-Turn Test Case

POST /api/v1/test-case/

datasetId, question, and groundTruth are required. status defaults to active; evaluations only run active test cases.

Request Content

Request Parameters

Field
Type
Required
Description

datasetId

string (uuid)

Yes

question

string

Yes

groundTruth

string

Yes

status

string

No

flaky

boolean

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 201

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Upload a CSV or Excel File to Add Test Cases

POST /api/v1/test-case/upload/

Upload as multipart/form-data; both file and dataset_id are required. The file must be .csv, .xlsx, or .xls, and the header row must contain the two columns question and ground_truth. Each row adds one single-turn test case. Use UTF-8 encoding for CSV. The response is an array of the newly added test cases.

Code Examples

Response Content

Status Code: 201

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Generate Single-Turn Test Cases from Knowledge Base Files

POST /api/v1/test-case/generate-from-file/

Specify the source with knowledgeBaseIds (expanded to all processed files of a supported type in the knowledge base) or knowledgeBaseFileIds; the two can be combined. Supported formats are txt, docx, pdf, md, markdown, and mdx. numTestCases is the total number of questions to generate this time (default 10). If the number of files or questions exceeds the platform limit, the API returns 400 and the message states the limit. Generation runs in the background, and the response is 202 with a jobId; if the same test set already has a generation job in progress, that job's jobId is returned and no duplicate is created. Credit: The organization's Credit is checked before submission, and 402 is returned if it is insufficient; agentops_test_gen is deducted based on the number of questions actually generated.

Request Content

Request Parameters

Field
Type
Required
Description

chatbotFileIds

array[string]

No

List of KnowledgeBaseFile IDs to generate test cases from (deprecated)

knowledgeBaseFileIds

array[string]

No

List of KnowledgeBaseFile IDs to generate test cases from

knowledgeBaseIds

array[string]

No

List of KnowledgeBase IDs to generate test cases from all their files

datasetId

string (uuid)

Yes

numTestCases

integer

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 202

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


List Test Case Generation Jobs

GET /api/v1/test-case/generation-jobs/

Lists the organization's single-turn and multi-turn generation jobs, sorted from newest to oldest. You can filter by datasetId and status.

Parameters

Parameter
Required
Type
Description

datasetId

❌

string

Only list generation jobs of this dataset

page

❌

integer

Page number, starting from 1; returns 404 if it exceeds the last page

pageSize

❌

integer

Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)

status

❌

string

Filter by status: pending / running / completed / partial_completed / failed

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Retrieve Test Case Generation Job Progress

GET /api/v1/test-case/generation-jobs/{jobId}/

status is one of pending, running, completed, partial_completed, or failed. generatedCount is the number of questions generated so far, and fileResults lists the result and failure reason for each file. Both single-turn and multi-turn generation jobs are queried through this endpoint and distinguished by jobType.

Parameters

Parameter
Required
Type
Description

jobId

✅

string

Generation job ID, i.e. the jobId returned by "Generate Single-Turn Test Cases from Knowledge Base Files"

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Update a Single-Turn Test Case

PUT /api/v1/test-case/{id}/

Parameters

Parameter
Required
Type
Description

id

✅

string

Test case ID

Request Content

Request Parameters

Field
Type
Required
Description

question

string

Yes

groundTruth

string

No

status

string (enum: active, inactive, pending)

No

active: Active ; inactive: Inactive ; pending: Pending Review;

flaky

boolean

No

Flaky test cases still run and keep their results, but are excluded from the pass rate.

Example Request Structure

Example Request Values

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Partially Update a Single-Turn Test Case

PATCH /api/v1/test-case/{id}/

To temporarily stop testing a question, set status to inactive. Evaluations skip it, and past evaluation results are kept.

Parameters

Parameter
Required
Type
Description

id

✅

string

Test case ID

Request Content

Request Parameters

Field
Type
Required
Description

question

string

No

groundTruth

string

No

status

string (enum: active, inactive, pending)

No

active: Active ; inactive: Inactive ; pending: Pending Review;

flaky

boolean

No

Flaky test cases still run and keep their results, but are excluded from the pass rate.

Example Request Structure

Example Request Values

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Delete a Single-Turn Test Case

DELETE /api/v1/test-case/{id}/

This also deletes the per-question results of this test case in past evaluations. To only pause testing, use status instead.

Parameters

Parameter
Required
Type
Description

id

✅

string

Test case ID

Code Examples

Response Body

Status Code
Description

204

No response body

4XX

Error response; X-Error-Code carries its code (see "Error codes").


Bulk delete single-turn test cases

POST /api/v1/test-case/bulk-delete/

Pass an array of test case IDs to delete as testCaseIds; success returns 204. The impact of deletion is the same as Delete a single-turn test case.

Request Content

Request Parameters

Field
Type
Required
Description

test_case_ids

array[string]

Yes

List of test case IDs to delete

Example Request Structure

Request Example Values

Code Examples

Response Body

Status Code
Description

204

No response body

4XX

Error response; X-Error-Code carries its code (see "Error codes").


List Multi-Turn Test Cases

GET /api/v1/multi-turn-test-case/

Use datasetId to filter by a single test set.

Parameters

Parameter
Required
Type
Description

dataset

❌

string

Same as datasetId. Lists only the multi-turn conversation test cases of this dataset

datasetId

❌

string

Lists only the multi-turn conversation test cases of this dataset

ordering

❌

string

Sort field: created_at or updated_at. Prefix with - for newest first (e.g. -updated_at). If not specified, results are sorted by creation time, newest first

page

❌

integer

Page number, starting from 1. Returns 404 if beyond the last page

pageSize

❌

integer

Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)

search

❌

string

Keyword. Matches the scenario or the expected outcome (expected_outcome) (partial match, case-insensitive; when multiple keywords are separated by spaces, all of them must match)

status

❌

string

Lists only multi-turn conversation test cases with this status. Allowed values: `active`: Active ; `inactive`: Inactive;

Code Examples

Response Body

Status Code: 200

Response Schema Example

Example Response Values


Create a multi-turn test case

POST /api/v1/multi-turn-test-case/

datasetId, scenario (the task scenario the simulated user should accomplish), and expectedOutcome (the outcome a successful conversation should reach, which is also the stop condition for the simulated conversation) are required; userDescription is optional and describes the simulated user's tone, background, and level of expertise.

Request

Request Parameters

Field
Type
Required
Description

datasetId

string (uuid)

Yes

scenario

string

Yes

expectedOutcome

string

Yes

userDescription

string

No

status

object

No

flaky

boolean

No

Request Schema Example

Request Example Values

Code Examples

Response

Status code: 201

Response Schema Example

Response Example Values


Generate Multi-Turn Test Cases from Knowledge Base Files

POST /api/v1/multi-turn-test-case/generate-from-file/

Specify the source files with knowledgeBaseFileIds; numTestCases defaults to 5 and allows at most 20. Generation runs in the background and returns 202 with a jobId; track it with Retrieve test case generation job progress. Credit: Same as single-turn generation; agentops_test_gen is deducted based on the number of questions actually generated.

Request Content

Request Parameters

Field
Type
Required
Description

datasetId

string (uuid)

Yes

knowledgeBaseFileIds

array[string]

Yes

List of KnowledgeBaseFile IDs to generate multi-turn test cases from

numTestCases

integer

No

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 202

Example Response Structure

Example Response Values


Retrieve a Specific Multi-Turn Test Case

GET /api/v1/multi-turn-test-case/{id}/

Parameters

Parameter
Required
Type
Description

id

✅

string

Multi-turn test case ID

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values


Update a Multi-Turn Test Case

PUT /api/v1/multi-turn-test-case/{id}/

Parameters

Parameter
Required
Type
Description

id

✅

string

Multi-turn test case ID

Request Content

Request Parameters

Field
Type
Required
Description

scenario

string

Yes

Task scenario the simulated user should accomplish

expectedOutcome

string

Yes

Outcome a successful conversation should reach, also used as the simulation stop condition

userDescription

string

No

Persona of the simulated user, such as tone, background and level of expertise

status

string (enum: active, inactive)

No

active: Active ; inactive: Inactive;

flaky

boolean

No

Flaky test cases still run and keep their results, but are excluded from the pass rate.

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values


Partially Update a Multi-Turn Test Case

PATCH /api/v1/multi-turn-test-case/{id}/

Parameters

Parameter
Required
Type
Description

id

✅

string

Multi-turn test case ID

Request Content

Request Parameters

Field
Type
Required
Description

scenario

string

No

Task scenario the simulated user should accomplish

expectedOutcome

string

No

Outcome a successful conversation should reach, also used as the simulation stop condition

userDescription

string

No

Persona of the simulated user, such as tone, background and level of expertise

status

string (enum: active, inactive)

No

active: Active ; inactive: Inactive;

flaky

boolean

No

Flaky test cases still run and keep their results, but are excluded from the pass rate.

Example Request Structure

Example Request Values

Code Examples

Response Content

Status Code: 200

Example Response Structure

Example Response Values


Delete a Multi-Turn Test Case

DELETE /api/v1/multi-turn-test-case/{id}/

Parameters

Parameter
Required
Type
Description

id

✅

string

Multi-turn test case ID

Code Examples

Response Body

Status Code
Description

204

No response body


List Available Evaluation Models

GET /api/v1/evaluations/evaluation-llms/

The model whose isDefault is true is the default evaluation model, used when no evaluationLlmId is specified while running an evaluation.

Code Examples

Response

Status code: 200

Response Schema

Response Example

Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Run a Single-Turn Evaluation

POST /api/v1/evaluations/evaluate/

datasetId and chatbotId are required; if evaluationLlmId is omitted, the default evaluation model is used. metricConfig toggles the built-in metrics with boolean values, and customMetrics accepts up to 10 custom metrics. Only test cases whose status is active are evaluated. The evaluation runs in the background, and the response returns 202 with an evaluationId. Credit: The organization's Credit is checked before submission and 402 is returned if it is insufficient; when the evaluation finishes, agentops_test_run and agentops_deepeval are deducted based on the number of completed test cases, and calls to the evaluation model are charged separately.

Request

Request Parameters

Field
Type
Required
Description

datasetId

string (uuid)

Yes

chatbotId

string (uuid)

Yes

evaluationLlmId

string (uuid)

No

name

string

No

description

string

No

metricConfig

object

No

customMetrics

array[DatasetEvaluationCustomMetric]

No

Request Schema

Request Example

Code Examples

Response

Status code: 202

Response Schema

Response Example

Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Run a Multi-Turn Evaluation

POST /api/v1/evaluations/multi-turn-evaluate/

Uses the multi-turn test cases in the dataset to simulate conversations between a user and the Agent, then scores them. maxTurns is the maximum number of turns the simulated user speaks (2–10, default 6), and language is the language of the simulated user (Traditional Chinese, Simplified Chinese, English, Japanese, Korean; default Traditional Chinese). Returns 400 if the dataset has no active multi-turn test cases. The response and Credit behavior are the same as Run a Single-Turn Evaluation.

Request

Request Parameters

Field
Type
Required
Description

datasetId

string (uuid)

Yes

chatbotId

string (uuid)

Yes

evaluationLlmId

string (uuid)

No

name

string

No

description

string

No

maxTurns

integer

No

minTurns

integer

No

turnFaithfulness

boolean

No

turnRelevancy

boolean

No

conversationCompleteness

boolean

No

language

object

No

customMetrics

array[MultiTurnCustomMetric]

No

Request Schema

Request Example

Code Examples

Response

Status code: 202

Response Schema

Response Example

Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


List Evaluations

GET /api/v1/evaluations/

You can filter by datasetId, chatbotId, and status.

Parameters

Parameter
Required
Type
Description

chatbotId

❌

string

Only list evaluations of this AI assistant

datasetId

❌

string

Only list evaluations that use this dataset

excludeBuiltIn

❌

string

When true, excludes data of official built-in Agents (locked instances created by renting from the Agent marketplace) and keeps only assistants built by the organization; copies created with "Copy as self-built" count as self-built. Omitting it or passing false is the same as not applying this filter.

ordering

❌

string

Sort field: created_at or updated_at; prefix with - for newest first (e.g., -updated_at). If not specified, results are sorted by creation time, newest first

page

❌

integer

Page number, starting from 1; returns 404 if beyond the last page

pageSize

❌

integer

Number of items per page; default 20, maximum 100 (values above 100 are treated as 100)

query

❌

string

Keyword, matched against the evaluation name, description, or ID (partial match)

search

❌

string

Keyword, matched against the evaluation name, dataset name, or AI assistant name (partial match, case-insensitive; when multiple keywords are separated by spaces, all must match)

status

❌

string

Only list evaluations with this status. Available values: `pending`: Pending evaluation ; `running`: Evaluating ; `stopping`: Stopping ; `completed`: Completed ; `partial_completed`: Partially completed ; `failed`: Evaluation failed;

Code Examples

Response

Status code: 200

Response Schema

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Retrieve Evaluation Result

GET /api/v1/evaluations/{id}/

status is one of pending, running, stopping, completed, partial_completed, or failed. While a run is in progress, check progressPercentage; once it finishes, check passRate, overallScore, and the per-metric metricScores. A run that ends with no completed test cases is marked failed.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response Content

Status Code: 200

Response Schema Example

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


List Per-Question Results of a Single-Turn Evaluation

GET /api/v1/evaluations/{id}/details/

Single-turn evaluations only; multi-turn evaluations return 400. The response is an array of all test case results and is not paginated.

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response Content

Status code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


List Per-Conversation Results of a Multi-Turn Evaluation

GET /api/v1/evaluations/{id}/multi-turn-results/

Multi-turn evaluations only; single-turn evaluations return 400. The response is paginated.

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Evaluation ID

page

❌

integer

Page number, starting from 1; a page beyond the last page returns 404

pageSize

❌

integer

Items per page. Default 20, maximum 100 (values above 100 are treated as 100)

Code Examples

Response Content

Status code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Cancel a Running Evaluation

POST /api/v1/evaluations/{id}/cancel/

Only evaluations whose status is running can be canceled; otherwise 400 is returned. After cancellation, status first becomes stopping and ends as partial_completed once stopped. Scores already calculated are kept, and completed test cases still consume Credit as usual. You can later run the remaining test cases with Resume an incomplete evaluation.

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response Content

Status code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Resume an incomplete evaluation

POST /api/v1/evaluations/{id}/resume/

Only evaluations whose status is partial_completed can be resumed; otherwise 400 is returned. Failed and not-yet-run test cases are rerun while completed results are kept, and 202 is returned; Credit is deducted only for completed cases that were not billed before.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response

Status Code: 202

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Export evaluation report

POST /api/v1/evaluations/{id}/export/

Only single-turn evaluations whose status is completed can be exported; multi-turn evaluations are not currently supported and return 400. The report is generated as an Excel file in the background and 201 is returned with the export record; then use Get latest export status to get the download URL.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response

Status Code: 201

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Get latest export status

GET /api/v1/evaluations/{id}/export-status/

Returns the most recent export record of this evaluation: status is pending, processing, completed, or failed; when it is completed, download the file from fileUrl. If this evaluation has no export records, 404 is returned.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response

Status Code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


List export records

GET /api/v1/evaluations/{id}/exports/

The response is an array of all export records of this evaluation, sorted from newest to oldest, without pagination.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response

Status Code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Edit evaluation name and description

PATCH /api/v1/evaluations/{id}/

Only name and description are accepted; other fields are ignored. An evaluation can be edited in any status.

Parameters

Parameter
Required
Type
Description

id

✅

string

Evaluation ID

Request Body

Request Parameters

Field
Type
Required
Description

name

string

No

description

string

No

Example Request Structure

Example Request Values

Code Examples

Response

Status Code: 200

Example Response Structure

Example Response Values

Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").


Delete evaluation

DELETE /api/v1/evaluations/{id}/

Parameters

Parameter Name
Required
Type
Description

id

✅

string

Evaluation ID

Code Examples

Response Content

Status code
Description

204

No response body

4XX

Error response; X-Error-Code carries its code (see "Error codes").


Last updated

Was this helpful?