> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/api/en/api-reference/ping-gu-he-ce-shi-ji.md).

# Evaluations and Test Sets

The evaluation and test set endpoints correspond to "Test Sets" and "Automated Testing" in the AgentOps back office, letting you batch-test Agents from your own systems. The full flow is:

1. **Create a test set**: Use `POST /api/v1/dataset/` to create an empty test set, or use `POST /api/v1/dataset/upload/` to upload a CSV/Excel file and create one in a single step.
2. **Add test cases**: For single-turn Q\&A, use `/api/v1/test-case/` (create one by one, upload a file, or generate from a knowledge base file); for multi-turn conversation scenarios, use `/api/v1/multi-turn-test-case/`. Generating from a knowledge base file runs in the background and returns `202` with a `jobId`; track it with [Retrieve test case generation job progress](#test-case-generation-jobs-retrieve).
3. **Run an evaluation**: Use `POST /api/v1/evaluations/evaluate/` (single-turn) or `POST /api/v1/evaluations/multi-turn-evaluate/` (multi-turn) to specify the test set and Agent. The response is `202` with an `evaluationId`.
4. **Read results and export**: Poll `GET /api/v1/evaluations/{id}/` until `status` becomes `completed`, `partial_completed`, or `failed`. Per-question results for single-turn evaluations are in `details`, and per-conversation results for multi-turn evaluations are in `multi-turn-results`. A completed single-turn evaluation can be exported as an Excel report with `export`, and the download URL is available from `export-status`.

**Required permissions**: The API Key calls the API as the member who created it. Listing and all write operations on `/dataset/`, `/test-case/`, and `/multi-turn-test-case/` require a role with the "Test Set permission"; listing and all write operations on `/evaluations/` require the "Automated Testing permission" (the AgentOps permissions in [Role Permissions](https://docs.maiagent.ai/org/roles/role-permission); owners always have them). Insufficient permissions return `403`. Members other than the owner can only see and evaluate Agents that their role is allowed to use.

**Credit**: Generating test cases from knowledge base files deducts the pricing item `agentops_test_gen` based on the number of questions actually generated. Running an evaluation deducts `agentops_test_run` and `agentops_deepeval` when it finishes, based on the number of completed test cases, and the evaluation model calls used for scoring are billed separately according to that model's usage. Before a generation or evaluation request is submitted, the system checks the organization's Credit and quota, and returns `402` if they are insufficient. Manual creation, file upload, and reading results do not deduct Credit; for unit prices, see [List pricing items](https://docs.maiagent.ai/api/en/api-reference/credit.md#credit-items-list).

For features and back-office operations, see the user manual pages [Test Set Management](https://docs.maiagent.ai/build/agent-ops/test-datasets) and [Automated Evaluation and Agent Monitoring](https://docs.maiagent.ai/build/agent-ops/evaluations).

### List Test Sets <a href="#dataset-list" id="dataset-list"></a>

GET `/api/v1/dataset/`

#### Parameters

| Parameter  | Required | Type    | Description                                                                                                                                         |
| ---------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `isActive` | ❌        | boolean | List only active (true) or inactive (false) datasets                                                                                                |
| `ordering` | ❌        | string  | Sort field: created\_at or updated\_at. Prefix with - for newest first (e.g. -updated\_at). If not specified, sorted by creation time, newest first |
| `page`     | ❌        | integer | Page number, starting from 1. Returns 404 if beyond the last page                                                                                   |
| `pageSize` | ❌        | integer | Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)                                                             |
| `query`    | ❌        | string  | Keyword; matches the dataset name, description, or ID (partial match)                                                                               |
| `search`   | ❌        | string  | Keyword; matches the dataset name or description (partial, case-insensitive match; when multiple keywords are separated by spaces, all must match)  |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/dataset/?isActive=true&ordering=example&page=1&pageSize=1&query=example&search=example" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/dataset/?isActive=true&ordering=example&page=1&pageSize=1&query=example&search=example", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/?isActive=true&ordering=example&page=1&pageSize=1&query=example&search=example"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/dataset/?isActive=true&ordering=example&page=1&pageSize=1&query=example&search=example", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Response Schema Example**

```typescript
{
  "count": integer
  "next"?: string (uri) // Optional
  "previous"?: string (uri) // Optional
  "results": [
    {
      "id": string (uuid)
      "name": string
      "description"?: string // Optional
      "organization": 
      {
        "id": string (uuid)
        "name": string
      }
      "createdBy": 
      {
        "id": string (uuid)
        "name": string
      }
      "isActive"?: boolean // Optional
      "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
      "singleTurnTestCaseCount": integer
      "multiTurnTestCaseCount": integer
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
  ]
}
```

**Example Response Values**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Response string",
      "description": "Response string",
      "organization": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "createdBy": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "isActive": false,
      "passPolicy": null,
      "singleTurnTestCaseCount": 456,
      "multiTurnTestCaseCount": 456,
      "createdAt": "Response string",
      "updatedAt": "Response string"
    }
  ]
}
```

***

### Create an empty test set <a href="#dataset-create" id="dataset-create"></a>

POST `/api/v1/dataset/`

{% hint style="info" %}
Test set names must be unique within an organization; a duplicate name returns `400`.
{% endhint %}

#### Request Content

**Request Parameters**

| Field            | Type                            | Required | Description                                                                                    |
| ---------------- | ------------------------------- | -------- | ---------------------------------------------------------------------------------------------- |
| name             | string                          | Yes      |                                                                                                |
| description      | string                          | No       |                                                                                                |
| passPolicy       | object (with 1 property: rules) | No       | Saved pass policy that pre-fills new evaluations; checked for shape only.                      |
| passPolicy.rules | array\[PassRule]                | Yes      | Checked in order: the first matching rule decides a question, and the last one is the default. |

**Example Request Structure**

```typescript
{
  "name": string
  "description"?: string // Optional
  "passPolicy"?:  // Saved pass policy that pre-fills new evaluations; checked for shape only. (Optional)
  {
    "rules": [ // Checked in order: the first matching rule decides a question, and the last one is the default.
      {
        "id": string
        "name": string
        "when": string // "always", or {"tags_any": [...]} / {"tags_all": [...]}.
        "metrics"?: [ // Result keys this rule runs; omitted runs every metric the evaluation enables. (Optional)
          string
        ]
        "verdict": { // What counts as pass: the tiered rule, or every listed gate metric passing.
        {
          "type": string (enum: tiered, gates) // * `tiered` - tiered
* `gates` - gates
          "gates"?: [ // Result keys (as stored in metric_scores) that must all pass; gates verdict only. (Optional)
            string
          ]
        }
        }
      }
    ]
  }
}
```

**Example Request Values**

```json
{
  "name": "Example name",
  "description": "Example string",
  "passPolicy": {
    "rules": [
      {
        "id": "Example string",
        "name": "Example name",
        "when": "Example string",
        "metrics": [
          "Example string"
        ],
        "verdict": {
          "type": "tiered",
          "gates": [
            "Example string"
          ]
        }
      }
    ]
  }
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/dataset/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example name",
    "description": "Example string",
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "name": "Example name",
    "description": "Example string",
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  };

axios.post("https://api.maiagent.ai/api/v1/dataset/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "name": "Example name",
    "description": "Example string",
    "passPolicy": {
        "rules": [
            {
                "id": "Example string",
                "name": "Example name",
                "when": "Example string",
                "metrics": [
                    "Example string"
                ],
                "verdict": {
                    "type": "tiered",
                    "gates": [
                        "Example string"
                    ]
                }
            }
        ]
    }
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/dataset/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'name' => 'Example name',
            'description' => 'Example string',
            'passPolicy' => [
                'rules' => [
                    [
                        'id' => 'Example string',
                        'name' => 'Example name',
                        'when' => 'Example string',
                        'metrics' => [
                            'Example string'
                        ],
                        'verdict' => [
                            'type' => 'tiered',
                            'gates' => [
                                'Example string'
                            ]
                        ]
                    ]
                ]
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 201**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // Optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // Optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Upload CSV or Excel to create a test set <a href="#dataset-upload-create" id="dataset-upload-create"></a>

POST `/api/v1/dataset/upload/`

{% hint style="info" %}
Upload as `multipart/form-data`: `file` and `name` are required, and `description` is optional. The file format is the same as [Upload CSV or Excel to add test cases](#test-case-upload-create); test set names must be unique within an organization.
{% endhint %}

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/dataset/upload/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -F "file=@example_file.csv" \
  -F "name=Example string" \
  -F "description=Example string"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');
const FormData = require('form-data');

// Create a FormData object
const formData = new FormData();
// Replace with the actual file
formData.append('file', /* actual file object */, 'example_file.csv');
formData.append('name', 'Example string');
formData.append('description', 'Example string');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    ...formData.getHeaders()
  }
};

axios.post("https://api.maiagent.ai/api/v1/dataset/upload/", formData, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/upload/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}

# Create the file and form data
files = {
    'file': ('example_file.csv', open('example_file.csv', 'rb'), 'text/csv'),
    'name': (None, 'Example string'),
    'description': (None, 'Example string')
}

response = requests.post(url, files=files, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/dataset/upload/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ],
        'multipart' => [
            [
                'name' => 'file',
                'contents' => fopen('example_file.csv', 'r'),
                'filename' => 'example_file.csv'
            ],
            [
                'name' => 'name',
                'contents' => 'Example string'
            ],
            [
                'name' => 'description',
                'contents' => 'Example string'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 201**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // Optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // Optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Generate a test set from a knowledge base file <a href="#dataset-generate-from-file-create" id="dataset-generate-from-file-create"></a>

POST `/api/v1/dataset/generate-from-file/`

{% hint style="info" %}
Creates a new test set from a single knowledge base file (`knowledgeBaseFileId`) and generates single-turn test cases; `numTestCases` ranges from 1 to 50 (default 10). Generation runs in the background, and the actual response is `202` with a `taskId`; it does not include the test set ID, and there is no progress to query. Once generation finishes, the new test set appears in the list. If you need to track progress, first create an empty test set and then use [Generate single-turn test cases from a knowledge base file](#test-case-generate-from-file-create). **Credit**: The organization's Credit is checked before submission, and `402` is returned if it is insufficient; `agentops_test_gen` is deducted based on the number of questions actually generated.
{% endhint %}

#### Request Content

**Request Parameters**

| Field               | Type          | Required | Description |
| ------------------- | ------------- | -------- | ----------- |
| name                | string        | Yes      |             |
| description         | string        | No       |             |
| chatbotFileId       | string (uuid) | No       |             |
| knowledgeBaseFileId | string (uuid) | No       |             |
| numTestCases        | integer       | No       |             |

**Example Request Structure**

```typescript
{
  "name": string
  "description"?: string // Optional
  "chatbotFileId"?: string (uuid) // Optional
  "knowledgeBaseFileId"?: string (uuid) // Optional
  "numTestCases"?: integer // Optional
}
```

**Example Request Values**

```json
{
  "name": "Example name",
  "description": "Example string",
  "chatbotFileId": "550e8400-e29b-41d4-a716-446655440000",
  "knowledgeBaseFileId": "550e8400-e29b-41d4-a716-446655440000",
  "numTestCases": 123
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/dataset/generate-from-file/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example name",
    "description": "Example string",
    "chatbotFileId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "name": "Example name",
    "description": "Example string",
    "chatbotFileId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
  };

axios.post("https://api.maiagent.ai/api/v1/dataset/generate-from-file/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/generate-from-file/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "name": "Example name",
    "description": "Example string",
    "chatbotFileId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/dataset/generate-from-file/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'name' => 'Example name',
            'description' => 'Example string',
            'chatbotFileId' => '550e8400-e29b-41d4-a716-446655440000',
            'knowledgeBaseFileId' => '550e8400-e29b-41d4-a716-446655440000',
            'numTestCases' => 123
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 202**

**Example Response Structure**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted, indicating the request has been accepted and queued for background execution
  "message": string // Acceptance message
  "taskId": string // Background task ID
}
```

**Example Response Values**

```json
{
  "status": "accepted",
  "message": "Response string",
  "taskId": "Response string"
}
```

***

### List knowledge base files available for test case generation <a href="#dataset-knowledge-base-files-retrieve" id="dataset-knowledge-base-files-retrieve"></a>

GET `/api/v1/dataset/knowledge-base-files/`

{% hint style="info" %}
Grouped by knowledge base, lists only fully processed files of type txt, docx, pdf, md, markdown, or mdx. `pageSize` is the maximum number of files returned per knowledge base (default 5, maximum 100); `search` filters by filename.
{% endhint %}

#### Parameters

| Parameter name | Required | Type    | Description                                                                  |
| -------------- | -------- | ------- | ---------------------------------------------------------------------------- |
| `pageSize`     | ❌        | integer | Maximum number of files returned per knowledge base (default 5, maximum 100) |
| `search`       | ❌        | string  | Filter by filename (case-insensitive partial match)                          |

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# Example API call (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/dataset/knowledge-base-files/?pageSize=1&search=example" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/dataset/knowledge-base-files/?pageSize=1&search=example", config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/knowledge-base-files/?pageSize=1&search=example"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/dataset/knowledge-base-files/?pageSize=1&search=example", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response body

**Status code: 200**

**Example response schema**

```typescript
{
  "knowledgeBases": [ // Knowledge bases with available files, sorted by name; knowledge bases without available files are excluded
    {
      "id": string (uuid)
      "name": string
      "files": 
      {
        "count": integer // Total number of matching files in this knowledge base
        "results": [ // The latest page_size files
          {
            "id": string (uuid)
            "filename": string
            "fileType": string
            "size": integer // File size in bytes
            "knowledgeBase": 
            {
              "id": string (uuid)
              "name": string
            }
          }
        ]
      }
    }
  ]
}
```

**Example response values**

```json
{
  "knowledgeBases": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Response string",
      "files": {
        "count": 456,
        "results": [
          {
            "id": "550e8400-e29b-41d4-a716-446655440000",
            "filename": "Response string",
            "fileType": "Response string",
            "size": 456,
            "knowledgeBase": {
              "id": "550e8400-e29b-41d4-a716-446655440000",
              "name": "Response string"
            }
          }
        ]
      }
    }
  ]
}
```

***

### Retrieve a specific test set <a href="#dataset-retrieve" id="dataset-retrieve"></a>

GET `/api/v1/dataset/{id}/`

#### Parameters

| Parameter | Required | Type   | Description |
| --------- | -------- | ------ | ----------- |
| `id`      | ✅        | string | Dataset ID  |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // Optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // Optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Retrieve test set with single-turn test cases <a href="#dataset-details-retrieve" id="dataset-details-retrieve"></a>

GET `/api/v1/dataset/{id}/details/`

{% hint style="info" %}
In addition to the test set fields, the response includes a `testCases` array containing only single-turn test cases; for multi-turn test cases, use `GET /api/v1/multi-turn-test-case/?datasetId={id}`.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description |
| --------- | -------- | ------ | ----------- |
| `id`      | ✅        | string | Dataset ID  |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/details/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/details/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/details/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/details/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // Optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // Optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
  "testCases": [ // All single-turn test cases of this test set, from oldest to newest by creation time; an empty array when there are none
    {
      "id": string (uuid)
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "question": string
      "groundTruth"?: string // Optional
      "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
      "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (Optional)
      "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (Optional)
      "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
      "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (Optional)
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
  ]
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "Response string",
  "updatedAt": "Response string",
  "testCases": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "question": "Response string",
      "groundTruth": "Response string",
      "status": "active",
      "creationMethod": "manual",
      "sourceFile": "Response string",
      "flaky": false,
      "tags": null,
      "createdAt": "Response string",
      "updatedAt": "Response string"
    }
  ]
}
```

***

### Update test set <a href="#dataset-update" id="dataset-update"></a>

PUT `/api/v1/dataset/{id}/`

#### Parameters

| Parameter | Required | Type   | Description |
| --------- | -------- | ------ | ----------- |
| `id`      | ✅        | string | Dataset ID  |

#### Request Content

**Request Parameters**

| Field            | Type                            | Required | Description                                                                                    |
| ---------------- | ------------------------------- | -------- | ---------------------------------------------------------------------------------------------- |
| name             | string                          | Yes      |                                                                                                |
| description      | string                          | No       |                                                                                                |
| isActive         | boolean                         | No       |                                                                                                |
| passPolicy       | object (with 1 property: rules) | No       | Saved pass policy that pre-fills new evaluations; checked for shape only. Null clears it.      |
| passPolicy.rules | array\[PassRule]                | Yes      | Checked in order: the first matching rule decides a question, and the last one is the default. |

**Example Request Structure**

```typescript
{
  "name": string
  "description"?: string // Optional
  "isActive"?: boolean // Optional
  "passPolicy"?:  // Saved pass policy that pre-fills new evaluations; checked for shape only. Null clears it. (Optional)
  {
    "rules": [ // Checked in order: the first matching rule decides a question, and the last one is the default.
      {
        "id": string
        "name": string
        "when": string // "always", or {"tags_any": [...]} / {"tags_all": [...]}.
        "metrics"?: [ // Result keys this rule runs; omitted runs every metric the evaluation enables. (Optional)
          string
        ]
        "verdict": { // What counts as pass: the tiered rule, or every listed gate metric passing.
        {
          "type": string (enum: tiered, gates) // * `tiered` - tiered
* `gates` - gates
          "gates"?: [ // Result keys (as stored in metric_scores) that must all pass; gates verdict only. (Optional)
            string
          ]
        }
        }
      }
    ]
  }
}
```

**Example Request Values**

```json
{
  "name": "Example name",
  "description": "Example string",
  "isActive": true,
  "passPolicy": {
    "rules": [
      {
        "id": "Example string",
        "name": "Example name",
        "when": "Example string",
        "metrics": [
          "Example string"
        ],
        "verdict": {
          "type": "tiered",
          "gates": [
            "Example string"
          ]
        }
      }
    ]
  }
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PUT "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example name",
    "description": "Example string",
    "isActive": true,
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  }'

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "name": "Example name",
    "description": "Example string",
    "isActive": true,
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  };

axios.put("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "name": "Example name",
    "description": "Example string",
    "isActive": True,
    "passPolicy": {
        "rules": [
            {
                "id": "Example string",
                "name": "Example name",
                "when": "Example string",
                "metrics": [
                    "Example string"
                ],
                "verdict": {
                    "type": "tiered",
                    "gates": [
                        "Example string"
                    ]
                }
            }
        ]
    }
}

response = requests.put(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->put("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'name' => 'Example name',
            'description' => 'Example string',
            'isActive' => true,
            'passPolicy' => [
                'rules' => [
                    [
                        'id' => 'Example string',
                        'name' => 'Example name',
                        'when' => 'Example string',
                        'metrics' => [
                            'Example string'
                        ],
                        'verdict' => [
                            'type' => 'tiered',
                            'gates' => [
                                'Example string'
                            ]
                        ]
                    ]
                ]
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // Optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // Optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (Optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Partially update test set <a href="#dataset-partial-update" id="dataset-partial-update"></a>

PATCH `/api/v1/dataset/{id}/`

#### Parameters

| Parameter | Required | Type   | Description |
| --------- | -------- | ------ | ----------- |
| `id`      | ✅        | string | Dataset ID  |

#### Request Content

**Request Parameters**

| Field            | Type                            | Required | Description                                                                                    |
| ---------------- | ------------------------------- | -------- | ---------------------------------------------------------------------------------------------- |
| name             | string                          | No       |                                                                                                |
| description      | string                          | No       |                                                                                                |
| isActive         | boolean                         | No       |                                                                                                |
| passPolicy       | object (with 1 property: rules) | No       | Saved pass policy that pre-fills new evaluations; checked for shape only. Null clears it.      |
| passPolicy.rules | array\[PassRule]                | Yes      | Checked in order: the first matching rule decides a question, and the last one is the default. |

**Example Request Structure**

```typescript
{
  "name"?: string // Optional
  "description"?: string // Optional
  "isActive"?: boolean // Optional
  "passPolicy"?:  // Saved pass policy that pre-fills new evaluations; checked for shape only. Null clears it. (Optional)
  {
    "rules": [ // Checked in order: the first matching rule decides a question, and the last one is the default.
      {
        "id": string
        "name": string
        "when": string // "always", or {"tags_any": [...]} / {"tags_all": [...]}.
        "metrics"?: [ // Result keys this rule runs; omitted runs every metric the evaluation enables. (Optional)
          string
        ]
        "verdict": { // What counts as pass: the tiered rule, or every listed gate metric passing.
        {
          "type": string (enum: tiered, gates) // * `tiered` - tiered
* `gates` - gates
          "gates"?: [ // Result keys (as stored in metric_scores) that must all pass; gates verdict only. (Optional)
            string
          ]
        }
        }
      }
    ]
  }
}
```

**Example Request Values**

```json
{
  "name": "Example name",
  "description": "Example string",
  "isActive": true,
  "passPolicy": {
    "rules": [
      {
        "id": "Example string",
        "name": "Example name",
        "when": "Example string",
        "metrics": [
          "Example string"
        ],
        "verdict": {
          "type": "tiered",
          "gates": [
            "Example string"
          ]
        }
      }
    ]
  }
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PATCH "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example name",
    "description": "Example string",
    "isActive": true,
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  }'

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "name": "Example name",
    "description": "Example string",
    "isActive": true,
    "passPolicy": {
      "rules": [
        {
          "id": "Example string",
          "name": "Example name",
          "when": "Example string",
          "metrics": [
            "Example string"
          ],
          "verdict": {
            "type": "tiered",
            "gates": [
              "Example string"
            ]
          }
        }
      ]
    }
  };

axios.patch("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "name": "Example name",
    "description": "Example string",
    "isActive": True,
    "passPolicy": {
        "rules": [
            {
                "id": "Example string",
                "name": "Example name",
                "when": "Example string",
                "metrics": [
                    "Example string"
                ],
                "verdict": {
                    "type": "tiered",
                    "gates": [
                        "Example string"
                    ]
                }
            }
        ]
    }
}

response = requests.patch(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->patch("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'name' => 'Example name',
            'description' => 'Example string',
            'isActive' => true,
            'passPolicy' => [
                'rules' => [
                    [
                        'id' => 'Example string',
                        'name' => 'Example name',
                        'when' => 'Example string',
                        'metrics' => [
                            'Example string'
                        ],
                        'verdict' => [
                            'type' => 'tiered',
                            'gates' => [
                                'Example string'
                            ]
                        ]
                    ]
                ]
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "name": string
  "description"?: string // optional
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "createdBy": 
  {
    "id": string (uuid)
    "name": string
  }
  "isActive"?: boolean // optional
  "passPolicy"?: object // Pre-fills the pass rules of a new evaluation of this test set; never read when a run starts. (optional)
  "singleTurnTestCaseCount": integer
  "multiTurnTestCaseCount": integer
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "response string",
  "description": "response string",
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "response string"
  },
  "createdBy": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "response string"
  },
  "isActive": false,
  "passPolicy": null,
  "singleTurnTestCaseCount": 456,
  "multiTurnTestCaseCount": 456,
  "createdAt": "response string",
  "updatedAt": "response string"
}
```

***

### Delete a test set <a href="#dataset-destroy" id="dataset-destroy"></a>

DELETE `/api/v1/dataset/{id}/`

{% hint style="info" %}
This cannot be undone. Deleting a test set also deletes all test cases in it, as well as the evaluations and results that were run with this test set.
{% endhint %}

#### Parameters

| Parameter Name | Required | Type   | Description |
| -------------- | -------- | ------ | ----------- |
| `id`           | ✅        | string | Dataset ID  |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X DELETE "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request payload
const data = null;

axios.delete("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.delete(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->delete("https://api.maiagent.ai/api/v1/dataset/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

| Status code | Description      |
| ----------- | ---------------- |
| 204         | No response body |

***

### List single-turn test cases <a href="#test-case-list" id="test-case-list"></a>

GET `/api/v1/test-case/`

{% hint style="info" %}
Use `datasetId` to filter by a single test set. You can also filter by `status` or `creationMethod` (`manual`, `file`, `generate`).
{% endhint %}

#### Parameters

| Parameter        | Required | Type    | Description                                                                                                                                                      |
| ---------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `creationMethod` | ❌        | string  | Only list test cases created with this method. Allowed values: \`manual\`: Manual ; \`file\`: File import ; \`generate\`: Generated;                             |
| `datasetId`      | ❌        | string  | Only list test cases from this dataset                                                                                                                           |
| `ordering`       | ❌        | string  | Sort field: created\_at or updated\_at. Prefix with - for newest first (e.g., -updated\_at). If not specified, results are sorted by creation time, newest first |
| `page`           | ❌        | integer | Page number, starting from 1; returns 404 if beyond the last page                                                                                                |
| `pageSize`       | ❌        | integer | Items per page. Defaults to 20, with a maximum of 100 (larger values are treated as 100)                                                                         |
| `query`          | ❌        | string  | Keyword matched against the question, ground truth, or ID (partial match)                                                                                        |
| `search`         | ❌        | string  | Keyword matched against the question or ground truth (partial, case-insensitive match; when multiple keywords are separated by spaces, all must match)           |
| `status`         | ❌        | string  | Only list test cases with this status. Allowed values: \`active\`: Active ; \`inactive\`: Inactive ; \`pending\`: Pending Review;                                |
| `tags`           | ❌        | string  | Comma-separated tags (e.g., out\_of\_scope,red\_team); only lists test cases that carry at least one of these tags                                               |

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/test-case/?creationMethod=file&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&query=example&search=example&status=active&tags=example" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/test-case/?creationMethod=file&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&query=example&search=example&status=active&tags=example", config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/?creationMethod=file&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&query=example&search=example&status=active&tags=example"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("An error occurred:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/test-case/?creationMethod=file&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&query=example&search=example&status=active&tags=example", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response schema example**

```typescript
{
  "count": integer
  "next"?: string (uri) // optional
  "previous"?: string (uri) // optional
  "results": [
    {
      "id": string (uuid)
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "question": string
      "groundTruth"?: string // optional
      "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (optional)
      "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (optional)
      "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (optional)
      "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (optional)
      "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (optional)
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
  ]
}
```

**Response example**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "question": "Response string",
      "groundTruth": "Response string",
      "status": "active",
      "creationMethod": "manual",
      "sourceFile": "Response string",
      "flaky": false,
      "tags": null,
      "createdAt": "Response string",
      "updatedAt": "Response string"
    }
  ]
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Create a single-turn test case <a href="#test-case-create" id="test-case-create"></a>

POST `/api/v1/test-case/`

{% hint style="info" %}
`datasetId`, `question`, and `groundTruth` are required. `status` defaults to `active`; evaluations run only `active` test cases.
{% endhint %}

#### Request

**Request parameters**

| Field       | Type           | Required | Description                                                                  |
| ----------- | -------------- | -------- | ---------------------------------------------------------------------------- |
| datasetId   | string (uuid)  | Yes      |                                                                              |
| question    | string         | Yes      |                                                                              |
| groundTruth | string         | Yes      |                                                                              |
| status      | string         | No       |                                                                              |
| flaky       | boolean        | No       |                                                                              |
| tags        | array\[string] | No       | Labels that pick the pass rule deciding this question; empty means untagged. |

**Request schema example**

```typescript
{
  "datasetId": string (uuid)
  "question": string
  "groundTruth": string
  "status"?: string // optional
  "flaky"?: boolean // optional
  "tags"?: [ // Labels that pick the pass rule deciding this question; empty means untagged. (optional)
    string
  ]
}
```

**Request example**

```json
{
  "datasetId": "550e8400-e29b-41d4-a716-446655440000",
  "question": "Example string",
  "groundTruth": "Example string",
  "status": "Example string",
  "flaky": true,
  "tags": [
    "Example string"
  ]
}
```

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/test-case/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "Example string",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  }'

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "Example string",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  };

axios.post("https://api.maiagent.ai/api/v1/test-case/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "Example string",
    "flaky": True,
    "tags": [
        "Example string"
    ]
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("An error occurred:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/test-case/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'datasetId' => '550e8400-e29b-41d4-a716-446655440000',
            'question' => 'Example string',
            'groundTruth' => 'Example string',
            'status' => 'Example string',
            'flaky' => true,
            'tags' => [
                'Example string'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 201**

**Response schema example**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "question": string
  "groundTruth"?: string // optional
  "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (optional)
  "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (optional)
  "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (optional)
  "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Response example**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "question": "Response string",
  "groundTruth": "Response string",
  "status": "active",
  "creationMethod": "manual",
  "sourceFile": "Response string",
  "flaky": false,
  "tags": null,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Upload a CSV or Excel file to add test cases <a href="#test-case-upload-create" id="test-case-upload-create"></a>

POST `/api/v1/test-case/upload/`

{% hint style="info" %}
Upload as `multipart/form-data`; both `file` and `dataset_id` are required. The file must be `.csv`, `.xlsx`, or `.xls`, and the header row must contain the `question` and `ground_truth` columns. Each row adds one single-turn test case. Use UTF-8 encoding for CSV files. The response is an array of the newly created test cases.
{% endhint %}

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/test-case/upload/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -F "file=@example_file.csv" \
  -F "dataset_id=550e8400-e29b-41d4-a716-446655440000"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');
const FormData = require('form-data');

// Create a FormData object
const formData = new FormData();
// Replace with an actual file
formData.append('file', /* actual file object */, 'example_file.csv');
formData.append('dataset_id', '550e8400-e29b-41d4-a716-446655440000');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    ...formData.getHeaders()
  }
};

axios.post("https://api.maiagent.ai/api/v1/test-case/upload/", formData, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/upload/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}

# Create file and form data
files = {
    'file': ('example_file.csv', open('example_file.csv', 'rb'), 'text/csv'),
    'dataset_id': (None, '550e8400-e29b-41d4-a716-446655440000')
}

response = requests.post(url, files=files, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("An error occurred:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/test-case/upload/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ],
        'multipart' => [
            [
                'name' => 'file',
                'contents' => fopen('example_file.csv', 'r'),
                'filename' => 'example_file.csv'
            ],
            [
                'name' => 'dataset_id',
                'contents' => '550e8400-e29b-41d4-a716-446655440000'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 201**

**Response schema example**

```typescript
[
  {
    "id": string (uuid)
    "dataset": 
    {
      "id": string (uuid)
      "name": string
    }
    "question": string
    "groundTruth"?: string // optional
    "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (optional)
    "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (optional)
    "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (optional)
    "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (optional)
    "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (optional)
    "createdAt": string (timestamp)
    "updatedAt": string (timestamp)
  }
]
```

**Response example**

```json
[
  {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "dataset": {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Response string"
    },
    "question": "Response string",
    "groundTruth": "Response string",
    "status": "active",
    "creationMethod": "manual",
    "sourceFile": "Response string",
    "flaky": false,
    "tags": null,
    "createdAt": "Response string",
    "updatedAt": "Response string"
  }
]
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Generate Single-Turn Test Cases from Knowledge Base Files <a href="#test-case-generate-from-file-create" id="test-case-generate-from-file-create"></a>

POST `/api/v1/test-case/generate-from-file/`

{% hint style="info" %}
Specify the source with `knowledgeBaseIds` (expanded to all processed files of a supported type in the knowledge base) or `knowledgeBaseFileIds`; the two can be combined. Supported formats are txt, docx, pdf, md, markdown, and mdx. `numTestCases` is the total number of questions to generate this time (default 10). If the number of files or questions exceeds the platform limit, the API returns `400` and the message states the limit. Generation runs in the background, and the response is `202` with a `jobId`; if the same test set already has a generation job in progress, that job's `jobId` is returned and no duplicate is created. **Credit**: The organization's Credit is checked before submission, and `402` is returned if it is insufficient; `agentops_test_gen` is deducted based on the number of questions actually generated.
{% endhint %}

#### Request Content

**Request Parameters**

| Field                | Type           | Required | Description                                                            |
| -------------------- | -------------- | -------- | ---------------------------------------------------------------------- |
| chatbotFileIds       | array\[string] | No       | List of KnowledgeBaseFile IDs to generate test cases from (deprecated) |
| knowledgeBaseFileIds | array\[string] | No       | List of KnowledgeBaseFile IDs to generate test cases from              |
| knowledgeBaseIds     | array\[string] | No       | List of KnowledgeBase IDs to generate test cases from all their files  |
| datasetId            | string (uuid)  | Yes      |                                                                        |
| numTestCases         | integer        | No       |                                                                        |

**Example Request Structure**

```typescript
{
  "chatbotFileIds"?: [ // List of KnowledgeBaseFile IDs to generate test cases from (deprecated) (Optional)
    string (uuid)
  ]
  "knowledgeBaseFileIds"?: [ // List of KnowledgeBaseFile IDs to generate test cases from (Optional)
    string (uuid)
  ]
  "knowledgeBaseIds"?: [ // List of KnowledgeBase IDs to generate test cases from all their files (Optional)
    string (uuid)
  ]
  "datasetId": string (uuid)
  "numTestCases"?: integer // Optional
}
```

**Example Request Values**

```json
{
  "chatbotFileIds": [
    "550e8400-e29b-41d4-a716-446655440000"
  ],
  "knowledgeBaseFileIds": [
    "550e8400-e29b-41d4-a716-446655440000"
  ],
  "knowledgeBaseIds": [
    "550e8400-e29b-41d4-a716-446655440000"
  ],
  "datasetId": "550e8400-e29b-41d4-a716-446655440000",
  "numTestCases": 123
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/test-case/generate-from-file/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "chatbotFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "chatbotFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
  };

axios.post("https://api.maiagent.ai/api/v1/test-case/generate-from-file/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/generate-from-file/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "chatbotFileIds": [
        "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseFileIds": [
        "550e8400-e29b-41d4-a716-446655440000"
    ],
    "knowledgeBaseIds": [
        "550e8400-e29b-41d4-a716-446655440000"
    ],
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "numTestCases": 123
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/test-case/generate-from-file/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'chatbotFileIds' => [
                '550e8400-e29b-41d4-a716-446655440000'
            ],
            'knowledgeBaseFileIds' => [
                '550e8400-e29b-41d4-a716-446655440000'
            ],
            'knowledgeBaseIds' => [
                '550e8400-e29b-41d4-a716-446655440000'
            ],
            'datasetId' => '550e8400-e29b-41d4-a716-446655440000',
            'numTestCases' => 123
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status Code: 202**

**Example Response Structure**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted, meaning the request was accepted and queued for background execution
  "message": string // Acceptance message
  "jobId": string (uuid) // Generation job ID. Use GET /api/v1/test-case/generation-jobs/{jobId}/ to check progress; if the same test set already has a generation job in progress, that job is returned
  "taskId": string // Background task ID
}
```

**Example Response Values**

```json
{
  "status": "accepted",
  "message": "Response string",
  "jobId": "550e8400-e29b-41d4-a716-446655440000",
  "taskId": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### List Test Case Generation Jobs <a href="#test-case-generation-jobs-list" id="test-case-generation-jobs-list"></a>

GET `/api/v1/test-case/generation-jobs/`

{% hint style="info" %}
Lists the organization's single-turn and multi-turn generation jobs, sorted from newest to oldest. You can filter by `datasetId` and `status`.
{% endhint %}

#### Parameters

| Parameter Name | Required | Type    | Description                                                                   |
| -------------- | -------- | ------- | ----------------------------------------------------------------------------- |
| `datasetId`    | ❌        | string  | Only list generation jobs for this dataset                                    |
| `page`         | ❌        | integer | Page number, starting from 1; returns 404 if beyond the last page             |
| `pageSize`     | ❌        | integer | Items per page, default 20, maximum 100 (values above 100 are treated as 100) |
| `status`       | ❌        | string  | Filter by status: pending / running / completed / partial\_completed / failed |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/test-case/generation-jobs/?datasetId=example&page=1&pageSize=1&status=example" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/test-case/generation-jobs/?datasetId=example&page=1&pageSize=1&status=example", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/generation-jobs/?datasetId=example&page=1&pageSize=1&status=example"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/test-case/generation-jobs/?datasetId=example&page=1&pageSize=1&status=example", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "count": integer
  "next"?: string (uri) // Optional
  "previous"?: string (uri) // Optional
  "results": [
    {
      "id": string (uuid)
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "status"?: string (enum: pending, running, completed, partial_completed, failed) // * `pending` - Pending
* `running` - Generating
* `completed` - Completed
* `partial_completed` - Partially completed
* `failed` - Generation failed (Optional)
      "jobType"?:  // Single-turn Q&A test data, multi-turn conversation test data, or AI-drafted expected answers for existing questions

* `single_turn` - Single turn
* `multi_turn` - Multi turn
* `expected_answer` - Expected answer drafting (Optional)
      {
      }
      "requestedNumTestCases"?: integer // Optional
      "totalFiles"?: integer // Optional
      "processedFiles"?: integer // Optional
      "generatedCount"?: integer // Optional
      "progressPercentage": number (double)
      "errorMessage"?: string // Error message recorded when the whole batch fails (Optional)
      "fileResults"?: object // Result for each file, such as [{'file_id', 'filename', 'status', 'generated', 'error'}] (Optional)
      "startedAt"?: string (timestamp) // Optional
      "completedAt"?: string (timestamp) // Optional
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
  ]
}
```

**Example Response Values**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "status": "pending",
      "jobType": {},
      "requestedNumTestCases": 456,
      "totalFiles": 456,
      "processedFiles": 456,
      "generatedCount": 456,
      "progressPercentage": 456,
      "errorMessage": "Response string",
      "fileResults": null,
      "startedAt": "Response string",
      "completedAt": "Response string",
      "createdAt": "Response string",
      "updatedAt": "Response string"
    }
  ]
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Retrieve Test Case Generation Job Progress <a href="#test-case-generation-jobs-retrieve" id="test-case-generation-jobs-retrieve"></a>

GET `/api/v1/test-case/generation-jobs/{jobId}/`

{% hint style="info" %}
`status` is `pending`, `running`, `completed`, `partial_completed`, or `failed`. `generatedCount` is the number of questions generated so far, and `fileResults` lists the result and failure reason for each file. Both single-turn and multi-turn generation jobs are queried through this endpoint and distinguished by `jobType`.
{% endhint %}

#### Parameters

| Parameter Name | Required | Type   | Description                                                                                    |
| -------------- | -------- | ------ | ---------------------------------------------------------------------------------------------- |
| `jobId`        | ✅        | string | Generation job ID, i.e. the jobId returned by "Generate test cases from a knowledge base file" |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/test-case/generation-jobs/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/test-case/generation-jobs/550e8400-e29b-41d4-a716-446655440000/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/generation-jobs/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/test-case/generation-jobs/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "status"?: string (enum: pending, running, completed, partial_completed, failed) // * `pending` - Pending
* `running` - Generating
* `completed` - Completed
* `partial_completed` - Partially completed
* `failed` - Generation failed (Optional)
  "jobType"?:  // Single-turn Q&A test data, multi-turn conversation test data, or AI-drafted expected answers for existing questions

* `single_turn` - Single turn
* `multi_turn` - Multi turn
* `expected_answer` - Expected answer drafting (Optional)
  {
  }
  "requestedNumTestCases"?: integer // Optional
  "totalFiles"?: integer // Optional
  "processedFiles"?: integer // Optional
  "generatedCount"?: integer // Optional
  "progressPercentage": number (double)
  "errorMessage"?: string // Error message recorded when the whole batch fails (Optional)
  "fileResults"?: object // Result for each file, such as [{'file_id', 'filename', 'status', 'generated', 'error'}] (Optional)
  "startedAt"?: string (timestamp) // Optional
  "completedAt"?: string (timestamp) // Optional
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "status": "pending",
  "jobType": {},
  "requestedNumTestCases": 456,
  "totalFiles": 456,
  "processedFiles": 456,
  "generatedCount": 456,
  "progressPercentage": 456,
  "errorMessage": "Response string",
  "fileResults": null,
  "startedAt": "Response string",
  "completedAt": "Response string",
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Update a Single-Turn Test Case <a href="#test-case-update" id="test-case-update"></a>

PUT `/api/v1/test-case/{id}/`

#### Parameters

| Parameter Name | Required | Type   | Description  |
| -------------- | -------- | ------ | ------------ |
| `id`           | ✅        | string | Test case ID |

#### Request Content

**Request Parameters**

| Field       | Type                                     | Required | Description                                                                             |
| ----------- | ---------------------------------------- | -------- | --------------------------------------------------------------------------------------- |
| question    | string                                   | Yes      |                                                                                         |
| groundTruth | string                                   | No       |                                                                                         |
| status      | string (enum: active, inactive, pending) | No       | `active`: Active ; `inactive`: Inactive ; `pending`: Pending Review;                    |
| flaky       | boolean                                  | No       | Flaky test cases still run and keep their results, but are excluded from the pass rate. |
| tags        | array\[string]                           | No       | Labels that pick the pass rule deciding this question; empty means untagged.            |

**Example Request Structure**

```typescript
{
  "question": string
  "groundTruth"?: string // Optional
  "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "tags"?: [ // Labels that pick the pass rule deciding this question; empty means untagged. (Optional)
    string
  ]
}
```

**Example Request Values**

```json
{
  "question": "Example string",
  "groundTruth": "Example string",
  "status": "active",
  "flaky": true,
  "tags": [
    "Example string"
  ]
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PUT "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  };

axios.put("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": True,
    "tags": [
        "Example string"
    ]
}

response = requests.put(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->put("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'question' => 'Example string',
            'groundTruth' => 'Example string',
            'status' => 'active',
            'flaky' => true,
            'tags' => [
                'Example string'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "question": string
  "groundTruth"?: string // Optional
  "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
  "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (Optional)
  "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "question": "Response string",
  "groundTruth": "Response string",
  "status": "active",
  "creationMethod": "manual",
  "sourceFile": "Response string",
  "flaky": false,
  "tags": null,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Partially Update a Single-Turn Test Case <a href="#test-case-partial-update" id="test-case-partial-update"></a>

PATCH `/api/v1/test-case/{id}/`

{% hint style="info" %}
To temporarily skip a question, change its `status` to `inactive`. Evaluations will skip it, and past evaluation results are kept.
{% endhint %}

#### Parameters

| Parameter Name | Required | Type   | Description  |
| -------------- | -------- | ------ | ------------ |
| `id`           | ✅        | string | Test case ID |

#### Request Content

**Request Parameters**

| Field       | Type                                     | Required | Description                                                                             |
| ----------- | ---------------------------------------- | -------- | --------------------------------------------------------------------------------------- |
| question    | string                                   | No       |                                                                                         |
| groundTruth | string                                   | No       |                                                                                         |
| status      | string (enum: active, inactive, pending) | No       | `active`: Active ; `inactive`: Inactive ; `pending`: Pending Review;                    |
| flaky       | boolean                                  | No       | Flaky test cases still run and keep their results, but are excluded from the pass rate. |
| tags        | array\[string]                           | No       | Labels that pick the pass rule deciding this question; empty means untagged.            |

**Example Request Structure**

```typescript
{
  "question"?: string // Optional
  "groundTruth"?: string // Optional
  "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "tags"?: [ // Labels that pick the pass rule deciding this question; empty means untagged. (Optional)
    string
  ]
}
```

**Example Request Values**

```json
{
  "question": "Example string",
  "groundTruth": "Example string",
  "status": "active",
  "flaky": true,
  "tags": [
    "Example string"
  ]
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PATCH "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": true,
    "tags": [
      "Example string"
    ]
  };

axios.patch("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "question": "Example string",
    "groundTruth": "Example string",
    "status": "active",
    "flaky": True,
    "tags": [
        "Example string"
    ]
}

response = requests.patch(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->patch("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'question' => 'Example string',
            'groundTruth' => 'Example string',
            'status' => 'active',
            'flaky' => true,
            'tags' => [
                'Example string'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "question": string
  "groundTruth"?: string // Optional
  "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
  "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (Optional)
  "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "question": "Response string",
  "groundTruth": "Response string",
  "status": "active",
  "creationMethod": "manual",
  "sourceFile": "Response string",
  "flaky": false,
  "tags": null,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Delete a Single-Turn Test Case <a href="#test-case-destroy" id="test-case-destroy"></a>

DELETE `/api/v1/test-case/{id}/`

{% hint style="info" %}
This also deletes the per-question results of this test case in past evaluations. To only pause testing, use `status` instead.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description  |
| --------- | -------- | ------ | ------------ |
| `id`      | ✅        | string | Test case ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X DELETE "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request payload
const data = null;

axios.delete("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.delete(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->delete("https://api.maiagent.ai/api/v1/test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

| Status Code | Description                                                        |
| ----------- | ------------------------------------------------------------------ |
| 204         | No response body                                                   |
| 4XX         | Error response; X-Error-Code carries its code (see "Error codes"). |

***

### Bulk delete single-turn test cases <a href="#test-case-bulk-delete-create" id="test-case-bulk-delete-create"></a>

POST `/api/v1/test-case/bulk-delete/`

{% hint style="info" %}
Pass an array of test case IDs to delete as `testCaseIds`; success returns `204`. The impact of deletion is the same as [Delete a single-turn test case](#test-case-destroy).
{% endhint %}

#### Request Content

**Request Parameters**

| Field           | Type           | Required | Description                     |
| --------------- | -------------- | -------- | ------------------------------- |
| test\_case\_ids | array\[string] | Yes      | List of test case IDs to delete |

**Example Request Structure**

```typescript
{
  "test_case_ids": [ // List of test case IDs to delete
    string (uuid)
  ]
}
```

**Request Example Values**

```json
{
  "test_case_ids": [
    "550e8400-e29b-41d4-a716-446655440000"
  ]
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/test-case/bulk-delete/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "test_case_ids": [
      "550e8400-e29b-41d4-a716-446655440000"
    ]
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "test_case_ids": [
      "550e8400-e29b-41d4-a716-446655440000"
    ]
  };

axios.post("https://api.maiagent.ai/api/v1/test-case/bulk-delete/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/test-case/bulk-delete/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "test_case_ids": [
        "550e8400-e29b-41d4-a716-446655440000"
    ]
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/test-case/bulk-delete/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'test_case_ids' => [
                '550e8400-e29b-41d4-a716-446655440000'
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

| Status Code | Description                                                        |
| ----------- | ------------------------------------------------------------------ |
| 204         | No response body                                                   |
| 4XX         | Error response; X-Error-Code carries its code (see "Error codes"). |

***

### List Multi-Turn Test Cases <a href="#multi-turn-test-case-list" id="multi-turn-test-case-list"></a>

GET `/api/v1/multi-turn-test-case/`

{% hint style="info" %}
Use `datasetId` to filter by a single test set.
{% endhint %}

#### Parameters

| Parameter   | Required | Type    | Description                                                                                                                                                                         |
| ----------- | -------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `dataset`   | ❌        | string  | Same as datasetId. Lists only the multi-turn conversation test cases of this dataset                                                                                                |
| `datasetId` | ❌        | string  | Lists only the multi-turn conversation test cases of this dataset                                                                                                                   |
| `ordering`  | ❌        | string  | Sort field: created\_at or updated\_at. Prefix with - for newest first (e.g. -updated\_at). If not specified, results are sorted by creation time, newest first                     |
| `page`      | ❌        | integer | Page number, starting from 1. Returns 404 if beyond the last page                                                                                                                   |
| `pageSize`  | ❌        | integer | Number of items per page. Default 20, maximum 100 (values above 100 are treated as 100)                                                                                             |
| `search`    | ❌        | string  | Keyword. Matches the scenario or the expected outcome (expected\_outcome) (partial match, case-insensitive; when multiple keywords are separated by spaces, all of them must match) |
| `status`    | ❌        | string  | Lists only multi-turn conversation test cases with this status. Allowed values: \`active\`: Active ; \`inactive\`: Inactive;                                                        |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/multi-turn-test-case/?dataset=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&search=example&status=active" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/multi-turn-test-case/?dataset=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&search=example&status=active", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/?dataset=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&search=example&status=active"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/multi-turn-test-case/?dataset=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&ordering=example&page=1&pageSize=1&search=example&status=active", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

**Status Code: 200**

**Response Schema Example**

```typescript
{
  "count": integer
  "next"?: string (uri) // Optional
  "previous"?: string (uri) // Optional
  "results": [
    {
      "id": string (uuid)
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "scenario": string // Task scenario the simulated user should accomplish
      "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
      "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
      "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
      "creationMethod": 
      {
      }
      "sourceFile": string // Name of the knowledge base file this multi-turn test case was generated from
      "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
  ]
}
```

**Example Response Values**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "scenario": "Response string",
      "expectedOutcome": "Response string",
      "userDescription": "Response string",
      "status": "active",
      "creationMethod": {},
      "sourceFile": "Response string",
      "flaky": false,
      "createdAt": "Response string",
      "updatedAt": "Response string"
    }
  ]
}
```

***

### Create a multi-turn test case <a href="#multi-turn-test-case-create" id="multi-turn-test-case-create"></a>

POST `/api/v1/multi-turn-test-case/`

{% hint style="info" %}
`datasetId`, `scenario` (the task scenario the simulated user should accomplish), and `expectedOutcome` (the outcome a successful conversation should reach, which is also the stop condition for the simulated conversation) are required; `userDescription` is optional and describes the simulated user's tone, background, and level of expertise.
{% endhint %}

#### Request

**Request Parameters**

| Field           | Type          | Required | Description |
| --------------- | ------------- | -------- | ----------- |
| datasetId       | string (uuid) | Yes      |             |
| scenario        | string        | Yes      |             |
| expectedOutcome | string        | Yes      |             |
| userDescription | string        | No       |             |
| status          | object        | No       |             |
| flaky           | boolean       | No       |             |

**Request Schema Example**

```typescript
{
  "datasetId": string (uuid)
  "scenario": string
  "expectedOutcome": string
  "userDescription"?: string // Optional
  "status"?:  // Optional
  {
  }
  "flaky"?: boolean // Optional
}
```

**Request Example Values**

```json
{
  "datasetId": "550e8400-e29b-41d4-a716-446655440000",
  "scenario": "Example string",
  "expectedOutcome": "Example string",
  "userDescription": "Example string",
  "status": {},
  "flaky": true
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/multi-turn-test-case/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": {},
    "flaky": true
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": {},
    "flaky": true
  };

axios.post("https://api.maiagent.ai/api/v1/multi-turn-test-case/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": {},
    "flaky": True
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/multi-turn-test-case/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'datasetId' => '550e8400-e29b-41d4-a716-446655440000',
            'scenario' => 'Example string',
            'expectedOutcome' => 'Example string',
            'userDescription' => 'Example string',
            'status' => new \stdClass(),
            'flaky' => true
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 201**

**Response Schema Example**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "scenario": string // Task scenario the simulated user should accomplish
  "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "creationMethod": 
  {
  }
  "sourceFile": string // Name of the knowledge base file this multi-turn test case was generated from
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Response Example Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "scenario": "Response string",
  "expectedOutcome": "Response string",
  "userDescription": "Response string",
  "status": "active",
  "creationMethod": {},
  "sourceFile": "Response string",
  "flaky": false,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Generate Multi-Turn Test Cases from Knowledge Base Files <a href="#multi-turn-test-case-generate-from-file-create" id="multi-turn-test-case-generate-from-file-create"></a>

POST `/api/v1/multi-turn-test-case/generate-from-file/`

{% hint style="info" %}
Specify the source files with `knowledgeBaseFileIds`; `numTestCases` defaults to 5 and allows at most 20. Generation runs in the background and returns `202` with a `jobId`; track it with [Retrieve test case generation job progress](#test-case-generation-jobs-retrieve). **Credit**: Same as single-turn generation; `agentops_test_gen` is deducted based on the number of questions actually generated.
{% endhint %}

#### Request Content

**Request Parameters**

| Field                | Type           | Required | Description                                                          |
| -------------------- | -------------- | -------- | -------------------------------------------------------------------- |
| datasetId            | string (uuid)  | Yes      |                                                                      |
| knowledgeBaseFileIds | array\[string] | Yes      | List of KnowledgeBaseFile IDs to generate multi-turn test cases from |
| numTestCases         | integer        | No       |                                                                      |

**Example Request Structure**

```typescript
{
  "datasetId": string (uuid)
  "knowledgeBaseFileIds": [ // List of KnowledgeBaseFile IDs to generate multi-turn test cases from
    string (uuid)
  ]
  "numTestCases"?: integer // Optional
}
```

**Example Request Values**

```json
{
  "datasetId": "550e8400-e29b-41d4-a716-446655440000",
  "knowledgeBaseFileIds": [
    "550e8400-e29b-41d4-a716-446655440000"
  ],
  "numTestCases": 123
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/multi-turn-test-case/generate-from-file/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "numTestCases": 123
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileIds": [
      "550e8400-e29b-41d4-a716-446655440000"
    ],
    "numTestCases": 123
  };

axios.post("https://api.maiagent.ai/api/v1/multi-turn-test-case/generate-from-file/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/generate-from-file/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "datasetId": "550e8400-e29b-41d4-a716-446655440000",
    "knowledgeBaseFileIds": [
        "550e8400-e29b-41d4-a716-446655440000"
    ],
    "numTestCases": 123
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/multi-turn-test-case/generate-from-file/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'datasetId' => '550e8400-e29b-41d4-a716-446655440000',
            'knowledgeBaseFileIds' => [
                '550e8400-e29b-41d4-a716-446655440000'
            ],
            'numTestCases' => 123
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status Code: 202**

**Example Response Structure**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted, indicating the request has been accepted and queued for background execution
  "message": string // Acceptance message
  "jobId": string (uuid) // Generation job ID; use GET /api/v1/test-case/generation-jobs/{jobId}/ to check progress. If the same test set already has a generation job in progress, that job is returned
  "taskId": string // Background task ID
}
```

**Example Response Values**

```json
{
  "status": "accepted",
  "message": "Response string",
  "jobId": "550e8400-e29b-41d4-a716-446655440000",
  "taskId": "Response string"
}
```

***

### Retrieve a Specific Multi-Turn Test Case <a href="#multi-turn-test-case-retrieve" id="multi-turn-test-case-retrieve"></a>

GET `/api/v1/multi-turn-test-case/{id}/`

#### Parameters

| Parameter | Required | Type   | Description             |
| --------- | -------- | ------ | ----------------------- |
| `id`      | ✅        | string | Multi-turn test case ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "scenario": string // Task scenario the simulated user should accomplish
  "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "creationMethod": 
  {
  }
  "sourceFile": string // Name of the knowledge base file this multi-turn test case was generated from
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "scenario": "Response string",
  "expectedOutcome": "Response string",
  "userDescription": "Response string",
  "status": "active",
  "creationMethod": {},
  "sourceFile": "Response string",
  "flaky": false,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Update a Multi-Turn Test Case <a href="#multi-turn-test-case-update" id="multi-turn-test-case-update"></a>

PUT `/api/v1/multi-turn-test-case/{id}/`

#### Parameters

| Parameter | Required | Type   | Description             |
| --------- | -------- | ------ | ----------------------- |
| `id`      | ✅        | string | Multi-turn test case ID |

#### Request Content

**Request Parameters**

| Field           | Type                            | Required | Description                                                                                |
| --------------- | ------------------------------- | -------- | ------------------------------------------------------------------------------------------ |
| scenario        | string                          | Yes      | Task scenario the simulated user should accomplish                                         |
| expectedOutcome | string                          | Yes      | Outcome a successful conversation should reach, also used as the simulation stop condition |
| userDescription | string                          | No       | Persona of the simulated user, such as tone, background and level of expertise             |
| status          | string (enum: active, inactive) | No       | `active`: Active ; `inactive`: Inactive;                                                   |
| flaky           | boolean                         | No       | Flaky test cases still run and keep their results, but are excluded from the pass rate.    |

**Example Request Structure**

```typescript
{
  "scenario": string // Task scenario the simulated user should accomplish
  "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
}
```

**Example Request Values**

```json
{
  "scenario": "Example string",
  "expectedOutcome": "Example string",
  "userDescription": "Example string",
  "status": "active",
  "flaky": true
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PUT "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": true
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": true
  };

axios.put("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": True
}

response = requests.put(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->put("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'scenario' => 'Example string',
            'expectedOutcome' => 'Example string',
            'userDescription' => 'Example string',
            'status' => 'active',
            'flaky' => true
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "scenario": string // Task scenario the simulated user should accomplish
  "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "creationMethod": 
  {
  }
  "sourceFile": string // Name of the knowledge base file this multi-turn test case was generated from
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "scenario": "Response string",
  "expectedOutcome": "Response string",
  "userDescription": "Response string",
  "status": "active",
  "creationMethod": {},
  "sourceFile": "Response string",
  "flaky": false,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Partially Update a Multi-Turn Test Case <a href="#multi-turn-test-case-partial-update" id="multi-turn-test-case-partial-update"></a>

PATCH `/api/v1/multi-turn-test-case/{id}/`

#### Parameters

| Parameter | Required | Type   | Description             |
| --------- | -------- | ------ | ----------------------- |
| `id`      | ✅        | string | Multi-turn test case ID |

#### Request Content

**Request Parameters**

| Field           | Type                            | Required | Description                                                                                |
| --------------- | ------------------------------- | -------- | ------------------------------------------------------------------------------------------ |
| scenario        | string                          | No       | Task scenario the simulated user should accomplish                                         |
| expectedOutcome | string                          | No       | Outcome a successful conversation should reach, also used as the simulation stop condition |
| userDescription | string                          | No       | Persona of the simulated user, such as tone, background and level of expertise             |
| status          | string (enum: active, inactive) | No       | `active`: Active ; `inactive`: Inactive;                                                   |
| flaky           | boolean                         | No       | Flaky test cases still run and keep their results, but are excluded from the pass rate.    |

**Example Request Structure**

```typescript
{
  "scenario"?: string // Task scenario the simulated user should accomplish (Optional)
  "expectedOutcome"?: string // Outcome a successful conversation should reach, also used as the simulation stop condition (Optional)
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
}
```

**Example Request Values**

```json
{
  "scenario": "Example string",
  "expectedOutcome": "Example string",
  "userDescription": "Example string",
  "status": "active",
  "flaky": true
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PATCH "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": true
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": true
  };

axios.patch("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "scenario": "Example string",
    "expectedOutcome": "Example string",
    "userDescription": "Example string",
    "status": "active",
    "flaky": True
}

response = requests.patch(url, json=data, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->patch("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'scenario' => 'Example string',
            'expectedOutcome' => 'Example string',
            'userDescription' => 'Example string',
            'status' => 'active',
            'flaky' => true
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "scenario": string // Task scenario the simulated user should accomplish
  "expectedOutcome": string // Outcome a successful conversation should reach, also used as the simulation stop condition
  "userDescription"?: string // Persona of the simulated user, such as tone, background and level of expertise (Optional)
  "status"?: string (enum: active, inactive) // * `active` - Active
* `inactive` - Inactive (Optional)
  "creationMethod": 
  {
  }
  "sourceFile": string // Name of the knowledge base file this multi-turn test case was generated from
  "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "scenario": "Response string",
  "expectedOutcome": "Response string",
  "userDescription": "Response string",
  "status": "active",
  "creationMethod": {},
  "sourceFile": "Response string",
  "flaky": false,
  "createdAt": "Response string",
  "updatedAt": "Response string"
}
```

***

### Delete a Multi-Turn Test Case <a href="#multi-turn-test-case-destroy" id="multi-turn-test-case-destroy"></a>

DELETE `/api/v1/multi-turn-test-case/{id}/`

#### Parameters

| Parameter | Required | Type   | Description             |
| --------- | -------- | ------ | ----------------------- |
| `id`      | ✅        | string | Multi-turn test case ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X DELETE "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request body (payload)
const data = null;

axios.delete("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.delete(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->delete("https://api.maiagent.ai/api/v1/multi-turn-test-case/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Body

| Status Code | Description      |
| ----------- | ---------------- |
| 204         | No response body |

***

### List Available Evaluation Models <a href="#evaluations-evaluation-llms-list" id="evaluations-evaluation-llms-list"></a>

GET `/api/v1/evaluations/evaluation-llms/`

{% hint style="info" %}
The model whose `isDefault` is `true` is the default evaluation model, used when no `evaluationLlmId` is specified while running an evaluation.
{% endhint %}

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# Example API call (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/evaluation-llms/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Before running, make sure to replace YOUR_API_KEY and check the request data.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/evaluation-llms/", config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/evaluation-llms/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/evaluation-llms/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response Schema**

```typescript
[
  {
    "id": string (uuid)
    "name": string
    "value": string
    "provider": string
    "isDefault": boolean
  }
]
```

**Response Example**

```json
[
  {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string",
    "value": "Response string",
    "provider": "Response string",
    "isDefault": false
  }
]
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Run Single-Turn Evaluation <a href="#evaluations-evaluate-create" id="evaluations-evaluate-create"></a>

POST `/api/v1/evaluations/evaluate/`

{% hint style="info" %}
`datasetId` and `chatbotId` are required. When `evaluationLlmId` is omitted, the default evaluation model is used. `metricConfig` toggles the built-in metrics with boolean values, and `customMetrics` accepts up to 10 custom metrics. Only test cases with the status `active` are evaluated. The evaluation runs in the background and the response returns `202` with an `evaluationId`. **Credit**: The organization's Credit is checked before submission, and `402` is returned if it is insufficient. When the evaluation finishes, `agentops_test_run` and `agentops_deepeval` are deducted based on the number of completed test cases; calls to the evaluation model are charged separately.
{% endhint %}

#### Request

**Request Parameters**

| Field            | Type                                  | Required | Description                                                                                     |
| ---------------- | ------------------------------------- | -------- | ----------------------------------------------------------------------------------------------- |
| datasetId        | string (uuid)                         | Yes      |                                                                                                 |
| chatbotId        | string (uuid)                         | Yes      |                                                                                                 |
| evaluationLlmId  | string (uuid)                         | No       |                                                                                                 |
| name             | string                                | No       |                                                                                                 |
| description      | string                                | No       |                                                                                                 |
| metricConfig     | object                                | No       |                                                                                                 |
| customMetrics    | array\[DatasetEvaluationCustomMetric] | No       |                                                                                                 |
| passPolicy       | object (1 property: rules)            | No       | Which metrics decide "passed"; omitted or null keeps the tiered rule over every enabled metric. |
| passPolicy.rules | array\[PassRule]                      | Yes      | Checked in order: the first matching rule decides a question, and the last one is the default.  |

**Example Request Schema**

```typescript
{
  "datasetId": string (uuid)
  "chatbotId": string (uuid)
  "evaluationLlmId"?: string (uuid) // Optional
  "name"?: string // Optional
  "description"?: string // Optional
  "metricConfig"?: object // Optional
  "customMetrics"?: [ // Optional
    {
      "name": string
      "criteria"?: string // Optional
      "evaluationSteps"?: [ // Optional
        string
      ]
      "threshold"?: number (double) // Optional
      "includeExpectedOutput"?: boolean // Also give the judge the test case's expected answer; questions without one are skipped. (Optional)
    }
  ]
  "passPolicy"?:  // Which metrics decide "passed"; omitted or null keeps the tiered rule over every enabled metric. (Optional)
  {
    "rules": [ // Checked in order: the first matching rule decides a question, and the last one is the default.
      {
        "id": string
        "name": string
        "when": string // "always", or {"tags_any": [...]} / {"tags_all": [...]}.
        "metrics"?: [ // Result keys this rule runs; omitted runs every metric the evaluation enables. (Optional)
          string
        ]
        "verdict": { // What counts as pass: the tiered rule, or every listed gate metric passing.
        {
          "type": string (enum: tiered, gates) // * `tiered` - tiered
* `gates` - gates
          "gates"?: [ // Result keys (as stored in metric_scores) that must all pass; gates verdict only. (Optional)
            string
          ]
        }
        }
      }
    ]
  }
}
```

**Example Request Values**

```json
{
  "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "evaluation_llm_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "name": "Evaluation Name",
  "description": "Evaluation Description",
  "metric_config": {
    "ground_truth": true,
    "task_completion": false,
    "tool_correctness": false,
    "guardrail": true,
    "summarize": false,
    "hallucination": true
  },
  "custom_metrics": [
    {
      "name": "Support Tone",
      "criteria": "The answer should be polite, empathetic, and professional in tone.",
      "threshold": 0.6
    }
  ]
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# Example API call (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/evaluations/evaluate/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "evaluation_llm_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "name": "Evaluation Name",
    "description": "Evaluation Description",
    "metric_config": {
      "ground_truth": true,
      "task_completion": false,
      "tool_correctness": false,
      "guardrail": true,
      "summarize": false,
      "hallucination": true
    },
    "custom_metrics": [
      {
        "name": "Support Tone",
        "criteria": "The answer should be polite, empathetic, and professional in tone.",
        "threshold": 0.6
      }
    ]
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "evaluation_llm_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "name": "Evaluation Name",
    "description": "Evaluation Description",
    "metric_config": {
      "ground_truth": true,
      "task_completion": false,
      "tool_correctness": false,
      "guardrail": true,
      "summarize": false,
      "hallucination": true
    },
    "custom_metrics": [
      {
        "name": "Support Tone",
        "criteria": "The answer should be polite, empathetic, and professional in tone.",
        "threshold": 0.6
      }
    ]
  };

axios.post("https://api.maiagent.ai/api/v1/evaluations/evaluate/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request failed:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/evaluate/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "evaluation_llm_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "name": "Evaluation Name",
    "description": "Evaluation Description",
    "metric_config": {
        "ground_truth": True,
        "task_completion": False,
        "tool_correctness": False,
        "guardrail": True,
        "summarize": False,
        "hallucination": True
    },
    "custom_metrics": [
        {
            "name": "Support Tone",
            "criteria": "The answer should be polite, empathetic, and professional in tone.",
            "threshold": 0.6
        }
    ]
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request failed:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/evaluations/evaluate/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'dataset_id' => '3fa85f64-5717-4562-b3fc-2c963f66afa6',
            'chatbot_id' => '3fa85f64-5717-4562-b3fc-2c963f66afa6',
            'evaluation_llm_id' => '3fa85f64-5717-4562-b3fc-2c963f66afa6',
            'name' => 'Evaluation Name',
            'description' => 'Evaluation Description',
            'metric_config' => [
                'ground_truth' => true,
                'task_completion' => false,
                'tool_correctness' => false,
                'guardrail' => true,
                'summarize' => false,
                'hallucination' => true
            ],
            'custom_metrics' => [
                [
                    'name' => 'Support Tone',
                    'criteria' => 'The answer should be polite, empathetic, and professional in tone.',
                    'threshold' => 0.6
                ]
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request failed: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 202**

**Example Response Schema**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted; indicates the request was accepted and queued for background execution
  "message": string // Acceptance message
  "evaluationId": string (uuid) // Evaluation ID; poll status via GET /api/v1/evaluations/{id}/ until it finishes
}
```

**Example Response Values**

```json
{
  "status": "accepted",
  "message": "Response string",
  "evaluationId": "550e8400-e29b-41d4-a716-446655440000"
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Run a Multi-Turn Evaluation <a href="#evaluations-multi-turn-evaluate-create" id="evaluations-multi-turn-evaluate-create"></a>

POST `/api/v1/evaluations/multi-turn-evaluate/`

{% hint style="info" %}
Uses the multi-turn test cases in the dataset to simulate conversations between a user and the Agent, then scores them. `maxTurns` is the maximum number of turns the simulated user speaks (2–10, default 6), and `language` is the language of the simulated user (`Traditional Chinese`, `Simplified Chinese`, `English`, `Japanese`, `Korean`; default `Traditional Chinese`). Returns `400` if the dataset has no `active` multi-turn test cases. The response and Credit behavior are the same as [Run a Single-Turn Evaluation](#evaluations-evaluate-create).
{% endhint %}

#### Request

**Request Parameters**

| Field                    | Type                          | Required | Description |
| ------------------------ | ----------------------------- | -------- | ----------- |
| datasetId                | string (uuid)                 | Yes      |             |
| chatbotId                | string (uuid)                 | Yes      |             |
| evaluationLlmId          | string (uuid)                 | No       |             |
| name                     | string                        | No       |             |
| description              | string                        | No       |             |
| maxTurns                 | integer                       | No       |             |
| minTurns                 | integer                       | No       |             |
| turnFaithfulness         | boolean                       | No       |             |
| turnRelevancy            | boolean                       | No       |             |
| conversationCompleteness | boolean                       | No       |             |
| language                 | object                        | No       |             |
| customMetrics            | array\[MultiTurnCustomMetric] | No       |             |

**Request Schema**

```typescript
{
  "datasetId": string (uuid)
  "chatbotId": string (uuid)
  "evaluationLlmId"?: string (uuid) // optional
  "name"?: string // optional
  "description"?: string // optional
  "maxTurns"?: integer // optional
  "minTurns"?: integer // optional
  "turnFaithfulness"?: boolean // optional
  "turnRelevancy"?: boolean // optional
  "conversationCompleteness"?: boolean // optional
  "language"?:  // optional
  {
  }
  "customMetrics"?: [ // optional
    {
      "name": string
      "criteria"?: string // optional
      "evaluationSteps"?: [ // optional
        string
      ]
      "threshold"?: number (double) // optional
    }
  ]
}
```

**Request Example**

```json
{
  "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "max_turns": 6,
  "min_turns": 2,
  "custom_metrics": [
    {
      "name": "Identity Check",
      "criteria": "The assistant confirms the user identity before sharing account details.",
      "threshold": 0.6
    }
  ]
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# Example API call (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/evaluations/multi-turn-evaluate/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "max_turns": 6,
    "min_turns": 2,
    "custom_metrics": [
      {
        "name": "Identity Check",
        "criteria": "The assistant confirms the user identity before sharing account details.",
        "threshold": 0.6
      }
    ]
  }'

# Before running, make sure to replace YOUR_API_KEY and check the request data.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request payload
const data = {
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "max_turns": 6,
    "min_turns": 2,
    "custom_metrics": [
      {
        "name": "Identity Check",
        "criteria": "The assistant confirms the user identity before sharing account details.",
        "threshold": 0.6
      }
    ]
  };

axios.post("https://api.maiagent.ai/api/v1/evaluations/multi-turn-evaluate/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/multi-turn-evaluate/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request payload
data = {
    "dataset_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "chatbot_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "max_turns": 6,
    "min_turns": 2,
    "custom_metrics": [
        {
            "name": "Identity Check",
            "criteria": "The assistant confirms the user identity before sharing account details.",
            "threshold": 0.6
        }
    ]
}

response = requests.post(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/evaluations/multi-turn-evaluate/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'dataset_id' => '3fa85f64-5717-4562-b3fc-2c963f66afa6',
            'chatbot_id' => '3fa85f64-5717-4562-b3fc-2c963f66afa6',
            'max_turns' => 6,
            'min_turns' => 2,
            'custom_metrics' => [
                [
                    'name' => 'Identity Check',
                    'criteria' => 'The assistant confirms the user identity before sharing account details.',
                    'threshold' => 0.6
                ]
            ]
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 202**

**Response Schema**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted, indicating the request was accepted and queued for background execution
  "message": string // Acceptance message
  "evaluationId": string (uuid) // Evaluation ID; poll status with GET /api/v1/evaluations/{id}/ until it finishes
}
```

**Response Example**

```json
{
  "status": "accepted",
  "message": "Response string",
  "evaluationId": "550e8400-e29b-41d4-a716-446655440000"
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### List evaluations <a href="#evaluations-list" id="evaluations-list"></a>

GET `/api/v1/evaluations/`

{% hint style="info" %}
Filter by `datasetId`, `chatbotId`, or `status`.
{% endhint %}

#### Parameters

| Parameter        | Required | Type    | Description                                                                                                                                                                                                                                                                                        |
| ---------------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `chatbotId`      | ❌        | string  | Only list evaluations of this AI assistant                                                                                                                                                                                                                                                         |
| `datasetId`      | ❌        | string  | Only list evaluations that use this dataset                                                                                                                                                                                                                                                        |
| `excludeBuiltIn` | ❌        | string  | When true, excludes data of official built-in Agents (locked instances created by renting from the Agent marketplace) and keeps only assistants built by the organization; copies created via "Copy as custom" count as self-built. Omitted or false behaves the same as not applying this filter. |
| `ordering`       | ❌        | string  | Sort field: created\_at or updated\_at; prefix with - for newest first (e.g. -updated\_at); defaults to newest first by creation time                                                                                                                                                              |
| `page`           | ❌        | integer | Page number, starting from 1. Returns 404 if beyond the last page                                                                                                                                                                                                                                  |
| `pageSize`       | ❌        | integer | Number of items per page, default 20, maximum 100 (values above 100 are treated as 100)                                                                                                                                                                                                            |
| `query`          | ❌        | string  | Keyword, matched against the evaluation name, description, or ID (partial match)                                                                                                                                                                                                                   |
| `search`         | ❌        | string  | Keyword, matched against the evaluation name, dataset name, or AI assistant name (partial match, case-insensitive; when multiple keywords are separated by spaces, all must match)                                                                                                                 |
| `status`         | ❌        | string  | Only list evaluations in this status. Available values: \`pending\`: Pending evaluation ; \`running\`: Evaluating ; \`stopping\`: Stopping ; \`completed\`: Completed ; \`partial\_completed\`: Partially completed ; \`failed\`: Evaluation failed;                                               |

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/?chatbotId=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&excludeBuiltIn=example&ordering=example&page=1&pageSize=1&query=example&search=example&status=completed" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/?chatbotId=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&excludeBuiltIn=example&ordering=example&page=1&pageSize=1&query=example&search=example&status=completed", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/?chatbotId=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&excludeBuiltIn=example&ordering=example&page=1&pageSize=1&query=example&search=example&status=completed"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/?chatbotId=550e8400-e29b-41d4-a716-446655440000&datasetId=550e8400-e29b-41d4-a716-446655440000&excludeBuiltIn=example&ordering=example&page=1&pageSize=1&query=example&search=example&status=completed", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response schema example**

```typescript
{
  "count": integer
  "next"?: string (uri) // Optional
  "previous"?: string (uri) // Optional
  "results": [
    {
      "id": string (uuid)
      "name": string
      "description": string
      "chatbot": 
      {
        "id": string (uuid)
        "name": string
      }
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "organization": 
      {
        "id": string (uuid)
        "name": string
      }
      "evaluationLlm": 
      {
        "id": string (uuid)
        "name": string
      }
      "status": 
      {
      }
      "queuedAt": string (timestamp) // Set while a pending evaluation is held back by the per-organization running cap and has not been sent to the workers yet; cleared when it is released. Deliberately not a status value: a held run is still pending to the frontend.
      "startedAt": string (timestamp)
      "completedAt": string (timestamp)
      "totalTestCases": integer
      "completedTestCases": integer
      "passedTestCases": integer // Number of test cases that passed
      "metricScores": object // Detailed score per evaluation metric, such as ground_truth, task_completion and tool_correctness
      "evaluationConfig": object
      "progressPercentage": number (double)
      "overallScore": number (double)
      "passRate": number (double)
      "averageExecutionTimeMs": integer
      "totalExecutionTimeMs": integer
      "chatbotConfig": string
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
      "insights": 
      {
        "summary": string
        "metricStats"?: object // Optional
        "recommendations"?: object // Optional
      }
    }
  ]
}
```

**Response example**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Response string",
      "description": "Response string",
      "chatbot": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "organization": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "evaluationLlm": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "status": {},
      "queuedAt": "Response string",
      "startedAt": "Response string",
      "completedAt": "Response string",
      "totalTestCases": 456,
      "completedTestCases": 456,
      "passedTestCases": 456,
      "metricScores": null,
      "evaluationConfig": null,
      "progressPercentage": 456,
      "overallScore": 456,
      "passRate": 456,
      "averageExecutionTimeMs": 456,
      "totalExecutionTimeMs": 456,
      "chatbotConfig": "Response string",
      "createdAt": "Response string",
      "updatedAt": "Response string",
      "insights": {
        "summary": "Response string",
        "metricStats": null,
        "recommendations": null
      }
    }
  ]
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Get evaluation result <a href="#evaluations-retrieve" id="evaluations-retrieve"></a>

GET `/api/v1/evaluations/{id}/`

{% hint style="info" %}
`status` is one of `pending`, `running`, `stopping`, `completed`, `partial_completed`, or `failed`. While running, check `progressPercentage`; after it ends, check `passRate`, `overallScore`, and the `metricScores` of each metric. If no test case completes, the evaluation ends as `failed`.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response schema example**

```typescript
{
  "id": string (uuid)
  "name": string
  "description": string
  "chatbot": 
  {
    "id": string (uuid)
    "name": string
  }
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "evaluationLlm": 
  {
    "id": string (uuid)
    "name": string
  }
  "status": 
  {
  }
  "queuedAt": string (timestamp) // Set while a pending evaluation is held back by the per-organization running cap and has not been sent to the workers yet; cleared when it is released. Deliberately not a status value: a held run is still pending to the frontend.
  "startedAt": string (timestamp)
  "completedAt": string (timestamp)
  "totalTestCases": integer
  "completedTestCases": integer
  "passedTestCases": integer // Number of test cases that passed
  "metricScores": object // Detailed score per evaluation metric, such as ground_truth, task_completion and tool_correctness
  "evaluationConfig": object
  "progressPercentage": number (double)
  "overallScore": number (double)
  "passRate": number (double)
  "averageExecutionTimeMs": integer
  "totalExecutionTimeMs": integer
  "chatbotConfig": string
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
  "insights": 
  {
    "summary": string
    "metricStats"?: object // Optional
    "recommendations"?: object // Optional
  }
  "metricSettings": [
    {
      "kind": string (enum: builtin, custom) // * `builtin` - builtin
* `custom` - custom
      "key": string // The metric_scores key of this metric in the run's case rows.
      "name": string
      "threshold": number (double)
      "passRule": string (enum: gte, lte) // * `gte` - gte
* `lte` - lte
      "criteria": string
      "evaluationSteps": [
        string
      ]
      "scoringSteps": [ // Multi-turn custom metrics only: the pinned judging steps, when they differ from evaluation_steps.
        string
      ]
    }
  ]
  "skippedNoExpectedAnswerCount": integer // Questions skipped because a metric of their rule needs an expected answer they lack.
  "passPolicy":  // The policy that decided "passed"; a run from before pass policies shows the implicit default. Null for multi-turn runs.
  {
    "rules": [ // Checked in order: the first matching rule decides a question, and the last one is the default.
      {
        "id": string
        "name": string
        "when": string // "always", or {"tags_any": [...]} / {"tags_all": [...]}.
        "metrics"?: [ // Result keys this rule runs; omitted runs every metric the evaluation enables. (Optional)
          string
        ]
        "verdict": { // What counts as pass: the tiered rule, or every listed gate metric passing.
        {
          "type": string (enum: tiered, gates) // * `tiered` - tiered
* `gates` - gates
          "gates"?: [ // Result keys (as stored in metric_scores) that must all pass; gates verdict only. (Optional)
            string
          ]
        }
        }
      }
    ]
  }
  "progress":  // Per-state case counts and timing for drawing the run's progress; unlike progress_percentage, errored and skipped cases count as finished.
  {
    "total": integer // Cases in this run, skipped ones included.
    "passed": integer // Cases that completed and passed.
    "failed": integer // Cases that completed without passing, hook-blocked answers included.
    "errored": integer // Cases that could not be answered or scored.
    "skipped": integer // Cases left out before the run started (single-turn only), e.g. no expected answer.
    "running": integer // Cases being answered or scored right now.
    "pending": integer // Cases not started yet.
    "finished": integer // passed + failed + errored + skipped.
    "percentage": number (double) // finished / total, 0 to 100; 0 for a run without cases.
    "elapsedMs": integer // Time since the current leg started (a resume restarts it), to completion or now; null before the run starts.
    "estimatedRemainingMs": integer // Estimated time until every case finishes, from the pace of this leg; null unless the run is running with at least one case completed in this leg and at least one case left.
  }
}
```

**Response example**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "chatbot": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "evaluationLlm": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "status": {},
  "queuedAt": "Response string",
  "startedAt": "Response string",
  "completedAt": "Response string",
  "totalTestCases": 456,
  "completedTestCases": 456,
  "passedTestCases": 456,
  "metricScores": null,
  "evaluationConfig": null,
  "progressPercentage": 456,
  "overallScore": 456,
  "passRate": 456,
  "averageExecutionTimeMs": 456,
  "totalExecutionTimeMs": 456,
  "chatbotConfig": "Response string",
  "createdAt": "Response string",
  "updatedAt": "Response string",
  "insights": {
    "summary": "Response string",
    "metricStats": null,
    "recommendations": null
  },
  "metricSettings": [
    {
      "kind": "builtin",
      "key": "Response string",
      "name": "Response string",
      "threshold": 456,
      "passRule": "gte",
      "criteria": "Response string",
      "evaluationSteps": [
        "Response string"
      ],
      "scoringSteps": [
        "Response string"
      ]
    }
  ],
  "skippedNoExpectedAnswerCount": 456,
  "passPolicy": {
    "rules": [
      {
        "id": "Response string",
        "name": "Response string",
        "when": "Response string",
        "metrics": [
          "Response string"
        ],
        "verdict": {
          "type": "tiered",
          "gates": [
            "Response string"
          ]
        }
      }
    ]
  },
  "progress": {
    "total": 456,
    "passed": 456,
    "failed": 456,
    "errored": 456,
    "skipped": 456,
    "running": 456,
    "pending": 456,
    "finished": 456,
    "percentage": 456,
    "elapsedMs": 456,
    "estimatedRemainingMs": 456
  }
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Get per-question results of a single-turn evaluation <a href="#evaluations-details-list" id="evaluations-details-list"></a>

GET `/api/v1/evaluations/{id}/details/`

{% hint style="info" %}
Only applies to single-turn evaluations; multi-turn evaluations return `400`. The response is an array of all test case results and is not paginated.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/details/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/details/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/details/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/details/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response schema example**

```typescript
[
  {
    "id": string (uuid)
    "datasetEvaluation": 
    {
      "id": string (uuid)
      "name": string
    }
    "testCase": 
    {
      "id": string (uuid)
      "dataset": 
      {
        "id": string (uuid)
        "name": string
      }
      "question": string
      "groundTruth"?: string // Optional
      "status"?: string (enum: active, inactive, pending) // * `active` - Active
* `inactive` - Inactive
* `pending` - Pending Review (Optional)
      "creationMethod"?: string (enum: manual, file, generate) // * `manual` - Manual
* `file` - File import
* `generate` - Generated (Optional)
      "sourceFile"?: string // Name of the CSV, Excel or knowledge base file this test case came from (Optional)
      "flaky"?: boolean // Flaky test cases still run and keep their results, but are excluded from the pass rate. (Optional)
      "tags"?: object // Labels that pick which pass rule decides this question. An empty list matches only rules that cover every question. (Optional)
      "createdAt": string (timestamp)
      "updatedAt": string (timestamp)
    }
    "status": 
    {
    }
    "chatbotResponse": string
    "context": [ // Enrich each context item with is_deleted status for referenced knowledge base files.
      object
    ]
    "metricScores": object
    "executionTimeMs": integer
    "executionTimeSeconds": number (double)
    "passed": boolean
    "matchedRuleId": string // Id of the pass rule in the run's policy that decided Passed; empty for runs before pass rules.
    "errorMessage": string // Error message recorded when the evaluation fails
    "startedAt": string (timestamp)
    "completedAt": string (timestamp)
    "retryCount": integer // Number of times this test case has been retried
    "lastAttemptAt": string (timestamp) // Time of the last evaluation attempt
    "createdAt": string (timestamp)
    "updatedAt": string (timestamp)
  }
]
```

**Response example**

```json
[
  {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "datasetEvaluation": {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "Response string"
    },
    "testCase": {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "dataset": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Response string"
      },
      "question": "Response string",
      "groundTruth": "Response string",
      "status": "active",
      "creationMethod": "manual",
      "sourceFile": "Response string",
      "flaky": false,
      "tags": null,
      "createdAt": "Response string",
      "updatedAt": "Response string"
    },
    "status": {},
    "chatbotResponse": "Response string",
    "context": [
      {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "name": "Example response name",
        "description": "Example response description"
      }
    ],
    "metricScores": null,
    "executionTimeMs": 456,
    "executionTimeSeconds": 456,
    "passed": false,
    "matchedRuleId": "Response string",
    "errorMessage": "Response string",
    "startedAt": "Response string",
    "completedAt": "Response string",
    "retryCount": 456,
    "lastAttemptAt": "Response string",
    "createdAt": "Response string",
    "updatedAt": "Response string"
  }
]
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### List Per-Conversation Results of a Multi-Turn Evaluation <a href="#evaluations-multi-turn-results-list" id="evaluations-multi-turn-results-list"></a>

GET `/api/v1/evaluations/{id}/multi-turn-results/`

{% hint style="info" %}
Multi-turn evaluations only; single-turn evaluations return `400`. The response is paginated.
{% endhint %}

#### Parameters

| Parameter Name | Required | Type    | Description                                                                   |
| -------------- | -------- | ------- | ----------------------------------------------------------------------------- |
| `id`           | ✅        | string  | Evaluation ID                                                                 |
| `page`         | ❌        | integer | Page number, starting from 1; a page beyond the last page returns 404         |
| `pageSize`     | ❌        | integer | Items per page. Default 20, maximum 100 (values above 100 are treated as 100) |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/multi-turn-results/?page=1&pageSize=1" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/multi-turn-results/?page=1&pageSize=1", config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/multi-turn-results/?page=1&pageSize=1"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/multi-turn-results/?page=1&pageSize=1", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

**Status code: 200**

**Example Response Structure**

```typescript
{
  "count": integer
  "next"?: string (uri) // Optional
  "previous"?: string (uri) // Optional
  "results": [
    {
      "id": string (uuid)
      "multiTurnTestCase": string (uuid)
      "scenario": string
      "expectedOutcome": string
      "flaky": boolean
      "status": 
      {
      }
      "turns": object // The whole simulated conversation; each turn is {role, content, retrieval_context}
      "metricScores": object
      "passed": boolean
      "errorMessage": string
      "startedAt": string (timestamp)
      "completedAt": string (timestamp)
      "createdAt": string (timestamp)
    }
  ]
}
```

**Example Response Values**

```json
{
  "count": 123,
  "next": "http://api.example.org/accounts/?page=4",
  "previous": "http://api.example.org/accounts/?page=2",
  "results": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "multiTurnTestCase": "550e8400-e29b-41d4-a716-446655440000",
      "scenario": "Response string",
      "expectedOutcome": "Response string",
      "flaky": false,
      "status": {},
      "turns": null,
      "metricScores": null,
      "passed": false,
      "errorMessage": "Response string",
      "startedAt": "Response string",
      "completedAt": "Response string",
      "createdAt": "Response string"
    }
  ]
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Cancel a Running Evaluation <a href="#evaluations-cancel-create" id="evaluations-cancel-create"></a>

POST `/api/v1/evaluations/{id}/cancel/`

{% hint style="info" %}
Only evaluations whose `status` is `running` can be canceled; otherwise `400` is returned. After cancellation, `status` first becomes `stopping` and ends as `partial_completed` once stopped. Scores already calculated are kept, and completed test cases are still charged Credits as usual. You can later use [Resume an Incomplete Evaluation](#evaluations-resume-create) to run the remainder.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# Example API call (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/cancel/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Before running, make sure to replace YOUR_API_KEY and verify the request data.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request payload
const data = null;

axios.post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/cancel/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/cancel/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.post(url, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/cancel/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status code: 200**

**Response Structure Example**

```typescript
{
  "id": string (uuid)
  "name": string
  "description": string
  "chatbot": 
  {
    "id": string (uuid)
    "name": string
  }
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "evaluationLlm": 
  {
    "id": string (uuid)
    "name": string
  }
  "status": 
  {
  }
  "queuedAt": string (timestamp) // Set while a pending evaluation is held back by the per-organization running cap and has not been sent to the workers yet; cleared when it is released. Deliberately not a status value: a held run is still pending to the frontend.
  "startedAt": string (timestamp)
  "completedAt": string (timestamp)
  "totalTestCases": integer
  "completedTestCases": integer
  "passedTestCases": integer // Number of test cases that passed
  "metricScores": object // Detailed score per evaluation metric, such as ground_truth, task_completion and tool_correctness
  "evaluationConfig": object
  "progressPercentage": number (double)
  "overallScore": number (double)
  "passRate": number (double)
  "averageExecutionTimeMs": integer
  "totalExecutionTimeMs": integer
  "chatbotConfig": string
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
  "insights": 
  {
    "summary": string
    "metricStats"?: object // optional
    "recommendations"?: object // optional
  }
}
```

**Response Example**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "chatbot": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "evaluationLlm": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "status": {},
  "queuedAt": "Response string",
  "startedAt": "Response string",
  "completedAt": "Response string",
  "totalTestCases": 456,
  "completedTestCases": 456,
  "passedTestCases": 456,
  "metricScores": null,
  "evaluationConfig": null,
  "progressPercentage": 456,
  "overallScore": 456,
  "passRate": 456,
  "averageExecutionTimeMs": 456,
  "totalExecutionTimeMs": 456,
  "chatbotConfig": "Response string",
  "createdAt": "Response string",
  "updatedAt": "Response string",
  "insights": {
    "summary": "Response string",
    "metricStats": null,
    "recommendations": null
  }
}
```

**Status code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Resume an incomplete evaluation <a href="#evaluations-resume-create" id="evaluations-resume-create"></a>

POST `/api/v1/evaluations/{id}/resume/`

{% hint style="info" %}
Only evaluations whose `status` is `partial_completed` can be resumed; otherwise `400` is returned. Failed and not-yet-run test cases are rerun while completed results are kept, and `202` is returned; Credit is deducted only for completed cases that were not billed before.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/resume/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request body (payload)
const data = null;

axios.post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/resume/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/resume/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.post(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/resume/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 202**

**Example Response Structure**

```typescript
{
  "status": string (enum: accepted) // * `accepted` - Always accepted; the request has been accepted and queued for background execution
  "message": string // Acceptance message
  "taskId": string // Background task ID
}
```

**Example Response Values**

```json
{
  "status": "accepted",
  "message": "Response string",
  "taskId": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Export evaluation report <a href="#evaluations-export-create" id="evaluations-export-create"></a>

POST `/api/v1/evaluations/{id}/export/`

{% hint style="info" %}
Only single-turn evaluations whose `status` is `completed` can be exported; multi-turn evaluations are not currently supported and return `400`. The report is generated as an Excel file in the background and `201` is returned with the export record; then use [Get latest export status](#evaluations-export-status-retrieve) to get the download URL.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X POST "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request body (payload)
const data = null;

axios.post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.post(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->post("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 201**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "userEmail": string (email)
  "datasetEvaluationId": string (uuid)
  "datasetEvaluationName": string
  "status": 
  {
  }
  "statusDisplay": string
  "fileUrl": string // Returned file URL
  "fileSize": integer // File size in bytes
  "errorMessage": string
  "processingDurationSeconds": number (double) // Calculated processing duration (seconds)
  "createdAt": string (timestamp)
  "startedAt": string (timestamp)
  "completedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "userEmail": "response@example.com",
  "datasetEvaluationId": "550e8400-e29b-41d4-a716-446655440000",
  "datasetEvaluationName": "Response string",
  "status": {},
  "statusDisplay": "Response string",
  "fileUrl": "Response string",
  "fileSize": 456,
  "errorMessage": "Response string",
  "processingDurationSeconds": 456,
  "createdAt": "Response string",
  "startedAt": "Response string",
  "completedAt": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Get latest export status <a href="#evaluations-export-status-retrieve" id="evaluations-export-status-retrieve"></a>

GET `/api/v1/evaluations/{id}/export-status/`

{% hint style="info" %}
Returns the most recent export record of this evaluation: `status` is `pending`, `processing`, `completed`, or `failed`; when it is `completed`, download the file from `fileUrl`. If this evaluation has no export records, `404` is returned.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export-status/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export-status/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export-status/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/export-status/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 200**

**Example Response Structure**

```typescript
{
  "id": string (uuid)
  "userEmail": string (email)
  "datasetEvaluationId": string (uuid)
  "datasetEvaluationName": string
  "status": 
  {
  }
  "statusDisplay": string
  "fileUrl": string // Returned file URL
  "fileSize": integer // File size in bytes
  "errorMessage": string
  "processingDurationSeconds": number (double) // Calculated processing duration (seconds)
  "createdAt": string (timestamp)
  "startedAt": string (timestamp)
  "completedAt": string (timestamp)
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "userEmail": "response@example.com",
  "datasetEvaluationId": "550e8400-e29b-41d4-a716-446655440000",
  "datasetEvaluationName": "Response string",
  "status": {},
  "statusDisplay": "Response string",
  "fileUrl": "Response string",
  "fileSize": 456,
  "errorMessage": "Response string",
  "processingDurationSeconds": 456,
  "createdAt": "Response string",
  "startedAt": "Response string",
  "completedAt": "Response string"
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### List export records <a href="#evaluations-exports-list" id="evaluations-exports-list"></a>

GET `/api/v1/evaluations/{id}/exports/`

{% hint style="info" %}
The response is an array of all export records of this evaluation, sorted from newest to oldest, without pagination.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X GET "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/exports/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

axios.get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/exports/", config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('An error occurred during the request:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/exports/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.get(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("An error occurred during the request:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->get("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/exports/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'An error occurred during the request: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 200**

**Example Response Structure**

```typescript
[
  {
    "id": string (uuid)
    "userEmail": string (email)
    "datasetEvaluationId": string (uuid)
    "datasetEvaluationName": string
    "status": 
    {
    }
    "statusDisplay": string
    "fileUrl": string // Returned file URL
    "fileSize": integer // File size in bytes
    "processingDurationSeconds": number (double) // Calculated processing duration (seconds)
    "createdAt": string (timestamp)
    "startedAt": string (timestamp)
    "completedAt": string (timestamp)
  }
]
```

**Example Response Values**

```json
[
  {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "userEmail": "response@example.com",
    "datasetEvaluationId": "550e8400-e29b-41d4-a716-446655440000",
    "datasetEvaluationName": "Response string",
    "status": {},
    "statusDisplay": "Response string",
    "fileUrl": "Response string",
    "fileSize": 456,
    "processingDurationSeconds": 456,
    "createdAt": "Response string",
    "startedAt": "Response string",
    "completedAt": "Response string"
  }
]
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Edit Evaluation Name and Description <a href="#evaluations-partial-update" id="evaluations-partial-update"></a>

PATCH `/api/v1/evaluations/{id}/`

{% hint style="info" %}
Only `name` and `description` are accepted; other fields are ignored. An evaluation can be edited in any status.
{% endhint %}

#### Parameters

| Parameter | Required | Type   | Description   |
| --------- | -------- | ------ | ------------- |
| `id`      | ✅        | string | Evaluation ID |

#### Request Body

**Request Parameters**

| Field       | Type   | Required | Description |
| ----------- | ------ | -------- | ----------- |
| name        | string | No       |             |
| description | string | No       |             |

**Request Schema Example**

```typescript
{
  "name"?: string // Optional
  "description"?: string // Optional
}
```

**Example Request Values**

```json
{
  "name": "Example name",
  "description": "Example string"
}
```

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X PATCH "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Example name",
    "description": "Example string"
  }'

# Replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY',
    'Content-Type': 'application/json'
  }
};

// Request body (payload)
const data = {
    "name": "Example name",
    "description": "Example string"
  };

axios.patch("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Response received successfully:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Request body (payload)
data = {
    "name": "Example name",
    "description": "Example string"
}

response = requests.patch(url, json=data, headers=headers)
try:
    print("Response received successfully:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->patch("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY',
            'Content-Type' => 'application/json'
        ],
        'json' => [
            'name' => 'Example name',
            'description' => 'Example string'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Response received successfully:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response

**Status Code: 200**

**Response Schema Example**

```typescript
{
  "id": string (uuid)
  "name": string
  "description": string
  "chatbot": 
  {
    "id": string (uuid)
    "name": string
  }
  "dataset": 
  {
    "id": string (uuid)
    "name": string
  }
  "organization": 
  {
    "id": string (uuid)
    "name": string
  }
  "evaluationLlm": 
  {
    "id": string (uuid)
    "name": string
  }
  "status": 
  {
  }
  "queuedAt": string (timestamp) // Set while a pending evaluation is held back by the per-organization running cap and has not been sent to the workers yet; cleared when it is released. Deliberately not a status value: a held run is still pending to the frontend.
  "startedAt": string (timestamp)
  "completedAt": string (timestamp)
  "totalTestCases": integer
  "completedTestCases": integer
  "passedTestCases": integer // Number of test cases that passed
  "metricScores": object // Detailed score per evaluation metric, such as ground_truth, task_completion and tool_correctness
  "evaluationConfig": object
  "progressPercentage": number (double)
  "overallScore": number (double)
  "passRate": number (double)
  "averageExecutionTimeMs": integer
  "totalExecutionTimeMs": integer
  "chatbotConfig": string
  "createdAt": string (timestamp)
  "updatedAt": string (timestamp)
  "insights": 
  {
    "summary": string
    "metricStats"?: object // Optional
    "recommendations"?: object // Optional
  }
}
```

**Example Response Values**

```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "Response string",
  "description": "Response string",
  "chatbot": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "dataset": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "organization": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "evaluationLlm": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Response string"
  },
  "status": {},
  "queuedAt": "Response string",
  "startedAt": "Response string",
  "completedAt": "Response string",
  "totalTestCases": 456,
  "completedTestCases": 456,
  "passedTestCases": 456,
  "metricScores": null,
  "evaluationConfig": null,
  "progressPercentage": 456,
  "overallScore": 456,
  "passRate": 456,
  "averageExecutionTimeMs": 456,
  "totalExecutionTimeMs": 456,
  "chatbotConfig": "Response string",
  "createdAt": "Response string",
  "updatedAt": "Response string",
  "insights": {
    "summary": "Response string",
    "metricStats": null,
    "recommendations": null
  }
}
```

**Status Code: 4XX - Error response; X-Error-Code carries its code (see "Error codes").**

***

### Delete evaluation <a href="#evaluations-destroy" id="evaluations-destroy"></a>

DELETE `/api/v1/evaluations/{id}/`

#### Parameters

| Parameter Name | Required | Type   | Description   |
| -------------- | -------- | ------ | ------------- |
| `id`           | ✅        | string | Evaluation ID |

#### Code Examples

{% tabs %}
{% tab title="Shell/Bash" %}

```bash
# API call example (Shell)
curl -X DELETE "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/" \
  -H "Authorization: Api-Key YOUR_API_KEY"

# Make sure to replace YOUR_API_KEY and verify the request data before running.
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const axios = require('axios');

// Set request headers
const config = {
  headers: {
    'Authorization': 'Api-Key YOUR_API_KEY'
  }
};

// Request body (payload)
const data = null;

axios.delete("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", data, config)
  .then(response => {
    console.log('Successfully retrieved the response:');
    console.log(response.data);
  })
  .catch(error => {
    console.error('Request error:');
    console.error(error.response?.data || error.message);
  });
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}


response = requests.delete(url, headers=headers)
try:
    print("Successfully retrieved the response:")
    print(response.json())
except Exception as e:
    print("Request error:", e)
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
require 'vendor/autoload.php';

$client = new GuzzleHttp\Client();

try {
    $response = $client->delete("https://api.maiagent.ai/api/v1/evaluations/550e8400-e29b-41d4-a716-446655440000/", [
        'headers' => [
            'Authorization' => 'Api-Key YOUR_API_KEY'
        ]
    ]);
    
    $data = json_decode($response->getBody(), true);
    echo "Successfully retrieved the response:\n";
    print_r($data);
} catch (Exception $e) {
    echo 'Request error: ' . $e->getMessage();
}
?>
```

{% endtab %}
{% endtabs %}

#### Response Content

| Status code | Description                                                        |
| ----------- | ------------------------------------------------------------------ |
| 204         | No response body                                                   |
| 4XX         | Error response; X-Error-Code carries its code (see "Error codes"). |

***


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.maiagent.ai/api/en/api-reference/ping-gu-he-ce-shi-ji.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
