> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/tech/en/ai-gateway/ai-gateway-openai-compatible.md).

# AI Gateway: OpenAI-Compatible API Integration

Connect to MaiAgent AI Gateway with the OpenAI SDK without changing your application logic：endpoints and base URL, Bearer authentication, model identifiers (aliases, custom models, fallback groups), r

This page is for engineers who want to call language models through MaiAgent AI Gateway, especially independent software vendors (ISVs) whose applications already use the OpenAI SDK. Gateway provides an OpenAI-compatible interface. Change the SDK's `base_url` to MaiAgent and `api_key` to a MaiAgent API Key to call models from different vendors available to your organization through a single entry point, without changing your calling logic.

For the safeguards Gateway provides across sources, models, data, responsibilities, and records, see [AI Gateway Security](/tech/en/ai-gateway/ai-gateway-security.md). This page covers integration only.

{% hint style="info" %}
The MaiAgent platform must enable AI Gateway for your organization. If it is not enabled, calls to inference endpoints with any key return the same `401` as an invalid key.
{% endhint %}

## Endpoint Overview <a href="#endpoints" id="endpoints"></a>

| Method | Path                           | Authentication | Purpose                                                    |
| ------ | ------------------------------ | -------------- | ---------------------------------------------------------- |
| `POST` | `/api/v1/chat/completions`     | Required       | Chat completions, with streaming and non-streaming support |
| `POST` | `/api/v1/embeddings`           | Required       | Text embeddings                                            |
| `POST` | `/api/v1/audio/transcriptions` | Required       | Speech-to-text (upload a complete audio file)              |
| `POST` | `/api/v1/audio/speech`         | Required       | Text-to-speech (return a complete audio file)              |
| `GET`  | `/api/v1/models`               | Optional       | List callable model identifiers and pricing                |

**Base URL** (the OpenAI SDK's `base_url`):

```
https://api.maiagent.ai/api/v1
```

The SDK automatically appends `/chat/completions`, `/embeddings`, `/audio/transcriptions`, `/audio/speech`, or `/models`. **Do not add** a trailing slash to the path.

## Authentication <a href="#authentication" id="authentication"></a>

Inference endpoints use Bearer authentication, matching the header the OpenAI SDK sends by default:

```
Authorization: Bearer <GATEWAY_API_KEY>
Content-Type: application/json
```

* Use a MaiAgent API Key, but Gateway endpoints only accept the `Bearer` prefix. The `Api-Key <金鑰>` format used by other MaiAgent APIs returns `401` here.
* All of the following return the same `401 invalid_api_key`, without revealing which check failed: a nonexistent, revoked, disabled, or expired key; a disabled member or user associated with the key; or an organization without AI Gateway enabled.
* Request bodies must be JSON. The only exception is speech-to-text, which requires an audio upload using `multipart/form-data` (the OpenAI SDK handles this automatically).

### Get a Gateway Key <a href="#get-api-key" id="get-api-key"></a>

In the MaiAgent admin console, go to <mark style="color:blue;">Organization Settings</mark> → <mark style="color:blue;">API Key Management</mark> → <mark style="color:blue;">Create API Key</mark>. The creation wizard has three steps: select a member, configure access and limits, and generate the key.

* Each key is associated with one member. Calls are recorded under that member and subject to their Credits allowance.
* In "Access and Limits," configure the models the key can call (you can only narrow the organization's available models; all are enabled by default), requests per minute (RPM), and tokens per minute (TPM).
* The complete key is displayed only once, when it is created. Save it to your secrets management system immediately.
* A member can hold multiple valid keys at the same time. To rotate a key without interrupting service, issue a new key first, deploy it, and then revoke the old key.
* The API Key Management page requires the "API Key Management" permission. Members without this permission can create their own keys through the avatar in the upper right → <mark style="color:blue;">Account</mark> → <mark style="color:blue;">API Keys</mark> (these also work with Gateway). Administrators configure model access and RPM/TPM on the API Key Management page.

In the console's left menu, <mark style="color:blue;">AI Gateway</mark> → <mark style="color:blue;">Integration Examples</mark> generates copyable cURL/Python/Node.js examples for your selected model. If a call fails, go to <mark style="color:blue;">AI Gateway</mark> → <mark style="color:blue;">Overview</mark> and run <mark style="color:blue;">Access Diagnostics</mark> to check the key, member, organization, and model access in turn. Your role needs AI Gateway permission to see the AI Gateway menu. For configuration instructions, see [Enable AI Gateway for a Specific Role](https://docs.maiagent.ai/org/role-permission#enable-ai-gateway-for-role) in the user manual.

## Minimal Examples <a href="#minimal-examples" id="minimal-examples"></a>

Store the key in an environment variable first, rather than hardcoding it:

```bash
export GATEWAY_API_KEY="<GATEWAY_API_KEY>"
```

{% tabs %}
{% tab title="OpenAI Python SDK" %}

```python
import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GATEWAY_API_KEY"],
    base_url="https://api.maiagent.ai/api/v1",
)

resp = client.chat.completions.create(
    model="<vendor>/<model>",  # An id from GET /models, or a fallback group slug
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

{% endtab %}

{% tab title="curl" %}

```bash
curl https://api.maiagent.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<vendor>/<model>",
    "messages": [
      { "role": "user", "content": "Hello!" }
    ]
  }'
```

{% endtab %}
{% endtabs %}

## Model Identifiers <a href="#model-ids" id="model-ids"></a>

The `model` field takes a **Gateway model identifier**, rather than a vendor's native ID. Every identifier must contain `/`; values without `/` return `400` immediately.

| What you want to call                        | What to put in `model`                                                    | Where to find it                                             |
| -------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------ |
| MaiAgent managed model                       | `{vendor}/{model}` (or a platform-configured alias)                       | The `id` returned by `GET /models`                           |
| Organization custom model (BYOK/self-hosted) | A system-generated Gateway model ID, in the form `{供應商 slug}/{模型名}-{隨機碼}` | "Gateway Model ID" on the console's AI Gateway → Models page |
| Fallback group                               | A system-generated group slug, in the form `maiagent/fallbacks-{隨機碼}`     | Console: AI Gateway → Fallback Model Groups                  |

Resolution order: platform aliases first, then `{vendor}/{model}`, organization custom models, and finally fallback groups. Platform models (aliases and `{vendor}/{model}`) are searched only within "available to the organization ∩ enabled for Gateway ∩ enabled for this key." Custom models and fallback groups must belong to your organization and are restricted by this key's access list. Models outside this scope and nonexistent models return the same `404 model_not_found`.

### List Callable Models with /models <a href="#list-models" id="list-models"></a>

```python
for m in client.models.list():
    print(m.id)
```

With a Gateway key, the list includes only models the key can actually call, accounting for the organization's blocked models and the key's access list. Without authentication, it returns the platform's public catalog without organization filtering. In addition to the standard OpenAI fields `id`, `object`, `created`, and `owned_by`, each entry includes:

| Field                          | Description                                                                                                                                                                                                                                                                                    |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`                         | Display name                                                                                                                                                                                                                                                                                   |
| `type`                         | `language` for language models; `transcription` (speech-to-text) or `speech` (text-to-speech) for audio models. For field differences, see "Speech-to-Text and Text-to-Speech"                                                                                                                 |
| `tags`                         | Capability tags: `tool-use`, `multimodal`, `reasoning`, `prompt-caching`, `streaming`. `reasoning` indicates that the model accepts `reasoning_effort`; see "Reasoning Effort"                                                                                                                 |
| `pricing`                      | Managed models: `input`/`output`, in `credits_per_1k_tokens` (includes the organization's Gateway service fee when called with a Gateway key; the unauthenticated public catalog shows base rates without the service fee). Custom models: `traffic_fee_percent`, in `percent_of_total_tokens` |
| `providers`                    | A list of backend providers that can serve the model. Each entry has a `slug`, which you can specify through `provider.order` below                                                                                                                                                            |
| `context_window`, `max_tokens` | Included only when configured                                                                                                                                                                                                                                                                  |

### Integration Behavior When an Organization Hides Chinese Models <a href="#hidden-china-models" id="hidden-china-models"></a>

When an organization enables "Hide Chinese Models," Chinese platform models are excluded based on the model origins registered on the platform, including models added later. This restriction applies to both Gateway's model catalog and actual calls:

* **Querying the catalog with a key**: `GET /models` excludes hidden platform models. The unauthenticated public catalog has no organization information, so this restriction does not apply.
* **Calling platform models directly**: Hidden models cannot be called through aliases or `{vendor}/{model}` and return `404 model_not_found`. Calling the same platform model with your own key does not bypass the organization's hiding setting.
* **Calling fallback groups**: Steps using hidden platform models are excluded first, and the remaining steps are used. If none remain, the call returns `404 model_not_found`, and the group is also excluded from the catalog retrieved with a key.
* **Organization custom models**: Gateway models created by the organization are not subject to this origin-based hiding setting, but still must meet the existing key access rules.

For example, an enterprise integration engineer may have configured a Chinese platform model as the default. After the organization enables hiding, the original identifier returns `404`. The engineer should retrieve `/models` again using the service's actual key, change the default to an available model in that list, and then resume calls. Do not use the unauthenticated public catalog to determine whether the organization can use a model.

{% hint style="info" %}
The hiding setting does not clear existing per-model block lists. After hiding is disabled, access still depends on the original organization enablement status and the key's access scope.
{% endhint %}

### Specify a Backend Provider <a href="#provider-order" id="provider-order"></a>

Multiple providers may serve the same model identifier (for example, the same model may be available from both its vendor and a cloud platform). To use a specific provider, add `provider.order` to the request body, using the `providers[].slug` returned by `/models`:

```python
resp = client.chat.completions.create(
    model="<vendor>/<model>",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"provider": {"order": ["<provider-slug>"]}},
)
```

Currently, only the first value in the list is used. If that provider does not have the model, the call returns `404 model_not_found`. Requests specifying `provider.order` do not resolve to fallback groups.

### Bring Your Own Key (BYOK) <a href="#byok" id="byok"></a>

Organization owners can register their own provider credentials in the console under <mark style="color:blue;">AI Gateway</mark> → <mark style="color:blue;">Model Providers</mark>. Once registered:

* When calling a managed model, if the organization has valid credentials for the same provider, Gateway uses the organization's credentials for the upstream call (the most recently updated entry if there are multiple entries for that provider). The caller's `model` does not need to change.
* You can also use these credentials to create "custom models" and call them with system-generated Gateway model IDs.
* If the upstream provider rejects the organization's credentials with 401/403, Gateway returns `502 upstream_auth_error`. The message indicates that the organization's BYOK credentials were rejected; update them in AI Gateway settings.

### Partially Update Provider Credentials <a href="#credential-partial-update" id="credential-partial-update"></a>

Integration applications can use `PATCH /api/v1/gateway/credentials/{id}/` to update their organization's provider credentials. This management interface requires login, AI Gateway enabled for the organization, AI Gateway access permission, and an organization owner as the writer. Its authentication serves a different purpose from the Gateway Bearer key for [inference endpoints](#authentication).

The request can contain `name`, `isActive`, or `credentials`. Within `credentials`, use camelCase field names such as `apiBase` and `apiKey`. Each credential field is handled as follows:

| Submitted value    | Update result                                                               |
| ------------------ | --------------------------------------------------------------------------- |
| Omitted field      | Keep the original value                                                     |
| Empty string `""`  | Keep the original value; indicates no change                                |
| New nonempty value | Replace the original value, then validate against the provider's rules      |
| `null`             | Clear an optional field. Required fields cannot be cleared and return `400` |

For example, `apiKey` for vLLM and OpenAI-compatible providers can be cleared with `null`, but `apiBase` is required. Anthropic's optional `apiBase` can be cleared with `null` to remove a custom endpoint from the credentials; the API Key remains required.

{% hint style="warning" %}
If an API Key is already stored, changing `apiBase` to a different nonempty endpoint requires you to re-enter `apiKey` at the same time, or explicitly clear the key with `null` when the provider allows it. Omitting the key or sending an empty string returns `400` with an error at `credentials.apiKey`, leaving the original settings unchanged. Changing only the name or resubmitting the same endpoint does not require re-entering the key.
{% endhint %}

Read responses provide only the masked preview `credentialsPreview`, never the raw credentials. For console instructions, see [Model Provider Settings](https://docs.maiagent.ai/build/ai-gateway#provider-credentials). This section covers only the update semantics for integration applications.

### Fallback Groups <a href="#fallback-groups" id="fallback-groups"></a>

A fallback group is an ordered list of models created by an organization owner in the console. The entire fallback chain is enabled only when `model` is set to the group slug. Calls to a single model do not switch automatically.

* **Switching conditions** (fixed by the system): timeouts, connection failures, or upstream 408/429/500/502/503/504/529 responses. Client errors such as invalid parameters or invalid keys do not trigger switching; the error is returned directly.
* **Streaming**: Gateway obtains the first content chunk from the upstream provider before starting its response, so it can switch on connection errors or rate limits before sending the first byte. Once content starts returning, it no longer switches.
* The response's `model` field retains the group slug you submitted. The actual model serving the request and whether fallback was triggered are recorded in the console's usage and request logs.
* Steps whose model data no longer exists, or whose organization custom model has been deleted (disabled), are skipped before the call. The primary model also changes to the first step in the list that is still callable. **This skipping rule does not apply to platform models that are merely disabled**.
* If the entire chain has no callable steps, inference returns `400 invalid_request_error` with code `fallback_group_unavailable`. Edit the group to add available models; do not retry it as a temporary upstream error.
* When reading a group through `GET /api/v1/gateway/fallback-groups/{id}/`, these invalid steps return `modelId: null`, `modelName: ""`, `managedModel: null`, and `customModel: null`. Reading the management interface requires login, AI Gateway enabled for the organization, and AI Gateway access permission. Only visible platform groups and your own organization's groups can be read.
* Organization-owned fallback groups appear in `/models` only if they still have callable steps. This rule does not mean that every platform virtual model disappears from the model catalog.

## Chat Completion Parameters <a href="#chat-completions-params" id="chat-completions-params"></a>

Fields accepted by `POST /chat/completions`:

| Field                          | Description                                                                                                                                                                                                     |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model`                        | Required; see the previous section                                                                                                                                                                              |
| `messages`                     | Required and nonempty. `role` can be `system`, `user`, `assistant`, or `tool`; `content` can be a string or a multimodal array of `text`/`image_url` entries. Supports `tool_calls`, `tool_call_id`, and `name` |
| `stream`                       | Default: `false`                                                                                                                                                                                                |
| `stream_options.include_usage` | Default: `false`; see "Streaming"                                                                                                                                                                               |
| `temperature`                  | Numeric value                                                                                                                                                                                                   |
| `max_tokens`                   | Minimum: 1                                                                                                                                                                                                      |
| `top_p`                        | 0 to 1                                                                                                                                                                                                          |
| `stop`                         | String or array of strings                                                                                                                                                                                      |
| `n`                            | 1 to 8; default: 1                                                                                                                                                                                              |
| `tools`                        | Only `type: "function"` is supported; function names can be up to 64 characters                                                                                                                                 |
| `tool_choice`                  | `auto`, `none`, `required`, or a specific function object                                                                                                                                                       |
| `parallel_tool_calls`          | Default: `true`                                                                                                                                                                                                 |
| `response_format`              | `type` is `text`, `json_object`, or `json_schema`; `json_schema` requires a `json_schema` object                                                                                                                |
| `reasoning_effort`             | `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`; applies only to models tagged `reasoning` in `/models`. See "Reasoning Effort"                                                                      |
| `provider`                     | MaiAgent extension field; see "Specify a Backend Provider"                                                                                                                                                      |

Notes:

* If the target model does not support `top_p` or `stop`, these parameters are skipped without an error.
* `tools` and `response_format: json_schema` cannot be used together; doing so returns `400`.
* When streaming in `json_schema` mode, the complete JSON is sent all at once in a single delta, rather than streamed character by character.
* `json_object` mode adds an instruction to the system message to "output only valid JSON."
* If a `json_object` response consists solely of a single Markdown `json` code block containing valid JSON, Gateway removes the code block markers before returning it. This applies to both regular and streaming responses.
* If there is explanatory text outside the block, the contents are not valid JSON, or truncation prevents the response from forming a complete valid JSON block, Gateway preserves the model's original response. Integrations must still parse and handle invalid JSON themselves; `json_object` alone cannot guarantee a parseable object every time.
* If a streaming `json_object` response starts with a backtick, Gateway waits for the entire response to finish before sending its contents. Other responses are sent chunk by chunk as usual. If the upstream provider fails midway, content already received but not yet sent is forwarded as-is before the error is reported.

The response is a standard `chat.completion` object with an `id` in the form `chatcmpl-...`. `model` returns the identifier you submitted. The response header `x-request-id` matches `id`; include it when reporting issues.

### Reasoning Effort (reasoning\_effort) <a href="#reasoning-effort" id="reasoning-effort"></a>

`reasoning_effort` specifies how deeply a model should reason using a common set of levels across providers, from weakest to strongest:

`none` < `minimal` < `low` < `medium` < `high` < `xhigh` < `max`

* **Accepted values**: Only the seven values above or `null` are accepted. Other values (including `off` and numeric token budgets) return `400 invalid_request`. `none` disables reasoning.
* **Omitted or `null`**: Gateway does not specify a level. If the platform has configured a default reasoning level for that model, it uses that level. Otherwise, it sends no reasoning parameter, leaving the provider's default to apply. An exception is a small number of OpenAI models (such as the GPT-5.6 series) that require a level to accept tool definitions; the platform sends `none` for these models.
* **No `reasoning` tag**: The model has no reasoning control. `reasoning_effort` is skipped without an error.
* **Conversion**: The level is first mapped to a level supported by the target model, then converted to the provider's native parameter (for example, OpenAI's `reasoning_effort`, Claude's effort or reasoning token budget, or Gemini's thinking budget). The platform maintains supported levels by model family. `/models` currently provides only the `reasoning` tag, rather than listing the supported levels for each model.

If the model does not support the level you submit, Gateway adjusts it as follows without returning an error:

| Situation                                              | What is actually sent                                                                                                                                                          |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Model supports the level                               | Sent unchanged                                                                                                                                                                 |
| Level is unsupported, but a weaker level exists        | Uses the closest level no stronger than requested (for example, if the model supports up to `high`, sending `max` results in `high`), so the adjustment does not increase cost |
| Requested level is below the model's minimum           | Uses the model's minimum level                                                                                                                                                 |
| Reasoning is requested (anything other than `none`)    | Will not be adjusted to disabled                                                                                                                                               |
| `none` is sent, but the model cannot disable reasoning | Sends no reasoning parameter and uses the provider's default. These models still reason and are billed for actual tokens                                                       |

{% hint style="info" %}
The same level represents different amounts of reasoning across models. Reassess latency and token usage when switching models; do not assume identical costs at the same level.
{% endhint %}

## Streaming <a href="#streaming" id="streaming"></a>

With `stream: true`, the response uses Server-Sent Events (`Content-Type: text/event-stream`). Each event is one line of `data: {chat.completion.chunk JSON}`, with `data: [DONE]` at normal completion. Simply iterate over it with the OpenAI SDK:

```python
stream = client.chat.completions.create(
    model="<vendor>/<model>",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

* With `include_usage: true`, an additional final event contains `usage` and an empty `choices` array. This option only controls whether usage is returned; it does not affect billing. Gateway records and bills usage regardless of this setting.
* Errors before streaming starts (authentication, quota, nonexistent model, upstream connection failure) are returned as regular HTTP errors, rather than SSE.
* If the upstream provider fails during streaming, the HTTP status is already `200`. Gateway sends a `data: {"error": {...}}` event and ends without sending `[DONE]`. Handle these events in your client.

## Embeddings <a href="#embeddings" id="embeddings"></a>

`POST /embeddings` works the same as OpenAI:

```python
emb = client.embeddings.create(
    model="<embedding-model>",
    input=["First text passage", "Second text passage"],
)
print(len(emb.data), len(emb.data[0].embedding))
```

* `input`: A nonempty string or a nonempty array of strings.
* `encoding_format`: `float` (default) or `base64`.
* `model`: Matches the identifiers of embedding models enabled for Gateway on the platform, then their names. Embedding models are not listed in `/models` and are not restricted by the API Key's model access list. Confirm available embedding model identifiers with MaiAgent.
* Gateway estimates `usage.prompt_tokens` using the `cl100k_base` tokenizer. This number is also used for billing.

## Speech-to-Text and Text-to-Speech <a href="#ai-gateway-audio" id="ai-gateway-audio"></a>

Gateway provides two OpenAI-compatible audio endpoints. To call them with the OpenAI SDK's `client.audio.transcriptions.create()` and `client.audio.speech.create()`, just change `base_url` and `api_key`. Both endpoints are synchronous: upload a complete audio file to get a transcript, or submit text to get a complete audio file. Streaming is not supported.

### Available Audio Models <a href="#audio-models" id="audio-models"></a>

Audio models are managed by the MaiAgent platform and always use the platform's provider credentials. BYOK and custom models are not supported. The public catalog currently lists the following models; use the result of `GET /models` with a key as the authoritative list of models you can actually call:

| `model`                  | `type`          | Purpose                                                                                                                                  |
| ------------------------ | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `openai/gpt-transcribe`  | `transcription` | Speech-to-text                                                                                                                           |
| `azure/speech-to-text`   | `transcription` | Speech-to-text                                                                                                                           |
| `openai/gpt-4o-mini-tts` | `speech`        | Text-to-speech. Voices: `alloy`, `ash`, `ballad`, `coral`, `echo`, `fable`, `onyx`, `nova`, `sage`, `shimmer`, `verse`, `marin`, `cedar` |

Audio models have different fields from language models in `/models`:

* `tags` is an empty array. There are no `providers`, `context_window`, or `max_tokens` fields.
* `pricing` is `{"price": "…", "unit": "…", "currency": "credits"}`, with `unit` set to `credits_per_second` (speech-to-text, per second) or `credits_per_1k_characters` (text-to-speech, per thousand characters). Prices include the organization's Gateway service fee when called with a Gateway key; the unauthenticated public catalog shows base rates.
* `speech` models also have `voices` (available voices) and `response_formats` (available audio formats; the first is the default).

{% hint style="warning" %}
**Keys restricted to specific models cannot use audio models.** The API Key's model access list currently allows only language models to be selected. If a key has an access list, `/models` excludes audio models and audio endpoint calls return `404 model_not_found`. To use audio, use a key without a model access list.
{% endhint %}

Organizations on character-based billing plans cannot use audio endpoints. Calls return `403` (`type` is `permission_error`, `code` is `billing_mode_not_supported`), and `/models` called with a key also excludes audio models.

### Speech-to-Text <a href="#audio-transcriptions" id="audio-transcriptions"></a>

`POST /audio/transcriptions`, with a `multipart/form-data` body:

```python
with open("meeting.mp3", "rb") as f:
    result = client.audio.transcriptions.create(
        model="openai/gpt-transcribe",
        file=f,
        language="zh",
    )
print(result.text)
```

| Field             | Required | Description                                                                                                                                                               |
| ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file`            | Yes      | Audio file, up to 25 MB (25 × 1024 × 1024 bytes). Format is determined by the filename extension; accepted formats are `mp3`, `mp4`, `mpeg`, `mpga`, `m4a`, `wav`, `webm` |
| `model`           | Yes      | Identifier of a model with `type` set to `transcription`                                                                                                                  |
| `language`        | No       | Audio language. If omitted, the model detects it automatically; see the rules below                                                                                       |
| `prompt`          | No       | Transcription prompt; sent only to OpenAI models                                                                                                                          |
| `temperature`     | No       | `0` to `1`; sent only to OpenAI models                                                                                                                                    |
| `response_format` | No       | `json` (default) or `text`; other values return `400`                                                                                                                     |
| `stream`          | No       | Only `false` is accepted; `true` returns `400`                                                                                                                            |

How `language` is handled depends on the model:

* **`openai/gpt-transcribe`**: Uses two-letter ISO-639-1 codes (such as `zh` and `en`); if you provide a full locale (such as `zh-TW`), only the language code before `-` is used.
* **`azure/speech-to-text`**: Accepts the two-letter codes `zh`, `en`, `ja`, `ko`, `th`, `vi`, `id`, `ms`, `fr`, `de`, `es`, `it`, `pt`, `ru`, and `hi` (converted to the corresponding locale, such as `zh` → `zh-TW`), or a full locale directly (such as `zh-TW`); other two-letter codes return `400`. If omitted, the language is automatically detected between `zh-TW` and `en-US`.

Response:

* `response_format: json`: `Content-Type: application/json`; `usage.seconds` is the number of seconds billed for this request.

  ```json
  {"text": "Transcript content", "usage": {"type": "duration", "seconds": 6}}
  ```
* `response_format: text`: `Content-Type: text/plain; charset=utf-8`; the body contains the transcript itself.

### Text-to-speech <a href="#audio-speech" id="audio-speech"></a>

`POST /audio/speech`, with a JSON body:

```python
response = client.audio.speech.create(
    model="openai/gpt-4o-mini-tts",
    voice="alloy",
    input="Welcome to MaiAgent.",
    response_format="mp3",
)
response.write_to_file("welcome.mp3")
```

| Field             | Required | Description                                                                                                         |
| ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------- |
| `model`           | Yes      | Model identifier with `type` set to `speech`                                                                        |
| `input`           | Yes      | Text to synthesize; must not be empty; maximum 4,096 characters                                                     |
| `voice`           | Yes      | Must be a value in the model's `voices`; unsupported values return `400` with a list of available values            |
| `response_format` | No       | Must be a value in the model's `response_formats` (currently `mp3` and `wav`); defaults to the model's first format |
| `speed`           | No       | Speech speed; the currently listed OpenAI models support `0.25`–`4.0`; values outside this range return `400`       |
| `instructions`    | No       | Tone instructions, supported only by OpenAI models; sending them to other models returns `400`                      |
| `stream_format`   | No       | Only accepts `audio`; SSE streaming is not supported                                                                |

The response body is a complete audio file, with `Content-Type` set to `audio/mpeg` (`mp3`) or `audio/wav` (`wav`).

### Common behavior of audio endpoints <a href="#audio-behavior" id="audio-behavior"></a>

* **Response headers**: Successful responses and errors that occur after field validation include `x-request-id`. Include it when reporting an issue.
* **`Accept` header**: The Gateway does not negotiate formats based on `Accept`. The OpenAI SDK's default `Accept: application/octet-stream` does not cause an error.
* **Checks before calling the provider**: Authentication, field validation, the model, parameters such as voice/format/speed/language, the billing plan, and Credits are all checked before calling the provider. If any check blocks the request, no Credits are deducted.
* **Upstream timeout**: 300 seconds for speech-to-text and 120 seconds for text-to-speech; the Gateway does not retry automatically.
* **Upstream errors**: Provider rejections with 400/413/415/422 return `400 upstream_invalid_request`; rejected platform credentials return `502 upstream_auth_error`; timeouts and other errors return `502 upstream_error`. The provider's original error message is never returned, and no Credits are deducted.
* **Error format**: The same as other Gateway endpoints; see “Error format”.

## Error format <a href="#errors" id="errors"></a>

All errors use the OpenAI format, and the OpenAI SDK converts them into the corresponding exceptions:

```json
{
  "error": {
    "message": "Model 'vendor/model' not found or not enabled",
    "type": "invalid_request_error",
    "code": "model_not_found",
    "param": null
  }
}
```

| HTTP | `type`                  | `code`                       | Common causes                                                                                                                            | Recommended action                                                             |
| ---- | ----------------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| 400  | `invalid_request_error` | `invalid_request`            | Field validation failed, `model` has no `/`, or `tools` is used with `json_schema`                                                       | Correct the request; do not retry                                              |
| 400  | `invalid_request_error` | `upstream_invalid_request`   | Upstream rejected the request with 400/413/415/422 (parameters, tool definitions, or input size exceed the model's limits)               | Adjust the request to the target model's limits                                |
| 400  | `invalid_request_error` | `fallback_group_unavailable` | The fallback group has no callable steps                                                                                                 | Edit the group and add available models                                        |
| 401  | `authentication_error`  | `invalid_api_key`            | The key is invalid, disabled, expired, or revoked, or Gateway is not enabled for the organization                                        | Check the key; use access diagnostics in the admin console to locate the issue |
| 402  | `billing_error`         | `insufficient_credits`       | Insufficient organization Credits, or the member/group quota is exhausted; blocked before calling upstream                               | Add Credits or adjust the quota                                                |
| 403  | `permission_error`      | `billing_mode_not_supported` | An organization on a character-based billing plan called an audio endpoint                                                               | Switch to a Credits billing plan                                               |
| 404  | `invalid_request_error` | `model_not_found`            | The model does not exist, is not available to the organization, is not enabled for this key, or is not offered by the specified provider | Use `/models` to confirm available IDs                                         |
| 429  | `rate_limit_error`      | `rate_limit_exceeded`        | The key's RPM or TPM limit was exceeded                                                                                                  | Retry with backoff                                                             |
| 502  | `server_error`          | `upstream_auth_error`        | Upstream rejected credentials (BYOK or platform credentials)                                                                             | BYOK: update the credentials; platform credentials: contact MaiAgent           |
| 502  | `server_error`          | `upstream_error`             | Upstream timeout, rate limiting, 5xx, or another error; the original upstream error message is not returned                              | Retry with backoff, or use a fallback group                                    |

## Usage and Credits billing <a href="#billing" id="billing"></a>

Each successful request deducts Credits from the organization. Chat completions use the token counts reported by upstream, while embeddings use token counts estimated by the Gateway (see “Embeddings”):

* **Managed models**: Billed at the model's input/output token rates (including prompt caching), plus the organization's Gateway service fee percentage. When calling `/models` with a Gateway key, `pricing` already includes the service fee; the unauthenticated public catalog shows base rates before the service fee.
* **Self-hosted models and BYOK credentials**: Your company pays upstream charges directly to the provider. MaiAgent only charges a traffic fee, calculated as the total tokens for that request (input + output + prompt cache writes and reads) × the organization's traffic rate, without charging LLM token rates. The user guide's [AI Gateway traffic fees for self-hosted models](https://docs.maiagent.ai/org/credits#ai-gateway-traffic-fee) explains how to verify this in the admin console.
* **Embeddings**: Billed by input token count, plus the Gateway service fee.
* **Speech-to-text**: Billed by the audio duration reported by the provider, rounded up to a whole second with a minimum of 1 second, multiplied by the model's per-second rate, plus the Gateway service fee. The response's `usage.seconds` is the billed duration. If the provider does not report a duration, the entire request fails with `502`; no transcript is returned and no Credits are deducted.
* **Text-to-speech**: Billed as the number of characters in `input` ÷ 1000 × the model's per-thousand-character rate, plus the Gateway service fee.
* **Quota checks for audio requests**: Before calling the provider, the organization must have enough Credits to cover the minimum cost (1 second for speech-to-text, or the full cost of this request's `input` for text-to-speech). Otherwise, `402` is returned. Failed audio requests do not deduct Credits; audio models use platform credentials and incur no traffic fee.
* **Interrupted streaming**: Tokens already generated are still billed if the client disconnects or an upstream error occurs during streaming.
* If organization Credits are insufficient or a member's quota is exhausted, the request returns `402` before calling upstream.

View usage by model, provider, and API Key in the admin console under <mark style="color:blue;">AI Gateway</mark> → <mark style="color:blue;">Overview</mark>. Credit deduction details are available in the Credits consumption records.

## Limits <a href="#limits" id="limits"></a>

* **RPM**: The key's maximum requests per minute applies to all MaiAgent APIs accessible with this key, not just the Gateway.
* **TPM**: The maximum tokens per minute applies only to Gateway chat completions and embeddings, not audio endpoints (audio endpoints are still subject to RPM limits). Token counts are known only after the upstream response, so a single request may exceed the limit once within that minute. Subsequent requests in the same minute return `429`, and the limit resets the next minute.
* No rate limit applies if RPM/TPM is not configured.
* The only OpenAI-compatible endpoints currently available are the five in the table above. OpenAI's Responses, Assistants, image endpoints, and other endpoints, as well as audio translation (`/audio/translations`) and real-time audio, are outside the Gateway's scope.

## Related documentation <a href="#related" id="related"></a>

* [AI Gateway security](/tech/en/ai-gateway/ai-gateway-security.md): Endpoint sources, model safeguards, data protection, and audit logs
* [Quickstart](/tech/en/api-integration/quickstart.md): Authentication for MaiAgent platform APIs (Agents, conversations, and knowledge bases)
* User guide [Credits](https://docs.maiagent.ai/org/credits): Credits balance, billing methods, and consumption records


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.maiagent.ai/tech/en/ai-gateway/ai-gateway-openai-compatible.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.