> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/tech/en/platform-development/ai-gateway-security.md). # AI Gateway Security The real risks enterprises face when using multiple models, as revealed by the ‘AI relay’ incident, and the safeguards and current limitations of MaiAgent AI Gateway across five dimensions: source, mo

Comparison between an unofficial AI relay and an enterprise AI Gateway — Both stand between you and the model; the difference is whether they can be verified

Over the past two years, enterprises have adopted more and more AI models, while integrations have become increasingly fragmented. OpenAI, Anthropic, and Google each have their own APIs. Development teams switch among providers to find the best and most affordable model for each task, and token usage has grown by multiples. This has naturally created a new role in the market: an intermediary layer between users and models that provides one entry point to multiple model providers, with unified keys and billing. There are legitimate versions of this role, such as the US-based OpenRouter, which offers more than 400 models through a single entry point. There are also gray-market versions. Because overseas models are subject to regional restrictions in China, many unofficial “AI relays” have emerged. Users can access Claude, ChatGPT, and Gemini without an overseas account or credit card by paying a small amount in Chinese yuan—often only 10% to 30% of the official price. The cost of that convenience is that all the risk is shifted to the user. Some customers have asked us: **Doesn’t MaiAgent also stand between users and models? Does that convenience come with the same security concerns?** That is a good question and deserves a serious answer. This article has two parts. Sections 1 and 2 explain what AI relays are and the methods unofficial operators use. Sections 3 and 4 explain what MaiAgent AI Gateway does differently while occupying the same position. ## 1. What Is an AI Relay? Let’s start with a technical term: API. You can think of it as a restaurant’s order window. A program sends a question through the window, the AI service processes it, and the answer returns through the same window. An “AI relay” adds another window between the user and the AI service. Users think they are connected to a single service, but their questions are actually forwarded to several models behind it. It is like a purchasing agent for the AI world. This business is particularly widespread in China for four structural reasons: * **Regional restrictions**: OpenAI does not support China, and Anthropic does not offer Claude directly in China. In 2025, Anthropic went further by barring overseas companies that are majority-owned by entities from unsupported regions such as China. * **High barriers to access**: Overseas accounts, payments, and identity verification are all cumbersome. Relays handle everything for users at once. * **Exploding token demand**: A token is a unit used to measure the amount of text processed by a model, roughly equivalent to one Chinese character. Chinese authorities announced in March 2026 that annual domestic token usage had grown by more than 1,000 times, and the AI Agent era is accelerating consumption further. * **Changing user priorities**: Developers have started asking, “Which model is best and most affordable for this task?” A single entry point to multiple models is naturally more convenient. It is worth noting that an “intermediary layer” is not inherently bad. Whether it belongs to the gray market has nothing to do with the country; it depends on whether it can be verified. Legitimate intermediary layers are already a major business. The industry recently reported that a payments company announced plans to acquire OpenRouter for more than US$7.5 billion. Centralizing all enterprise AI calls through a gateway, which then determines which model receives each call, is a natural trend as AI becomes infrastructure. The issue is not whether there is an intermediary. It is whether that intermediary can be trusted and verified. ## 2. How Unofficial Relays Work Interviews by Chinese media with unofficial operators and users have revealed this industry chain in considerable detail.

How unofficial AI relays operate and the three risks they create — Figure 1: How unofficial AI relays operate and the three risks borne by users

### The Barrier to Entry Is Lower Than Expected An operator using a pseudonym described the most common way to get started to Nanjing News: rent an overseas cloud server, download open-source AI gateway software, prepare a batch of overseas model accounts, add ready-made scripts, and connect a payment system. That is enough to open for business. Operators use trial offers and low-cost monthly plans from different platforms to attract users, then direct them to WeChat groups or their own websites to add credit. Some operators run three services at once, selling access to Claude, ChatGPT, and enterprise services separately. ### Low Prices Come from “Account Pools” Claude has fixed official pricing, yet some relays charge only 10% to 30% of that price. How can they be so cheap? The answer is account pools. Operators register or acquire large numbers of overseas accounts and maintain them automatically with scripts. Each time a user submits a question, the system selects an account from the pool that still works. If that account is suspended, it moves on to the next one. Where do the accounts in the pool come from? Researchers have identified sources such as new-user promotions, other people’s unused credits, and enterprise or education discounts. None of these sources can be backed by a contract. ### Three Risks Borne by Users **The service can disappear without warning.** One user bought a plan costing RMB 25 per month that claimed to provide a daily allowance of US$120. About 20 days later, a mass suspension wave hit and the store simply vanished. After switching to another provider, the monthly fee rose to three or four times the original price because, according to the operator, “the upstream provider had suffered a major disruption.” China began a crackdown at the end of May 2026, and many operators shut down. **You do not know which model you are actually using.** A German research team investigated 17 unofficial relay services. It used test prompts to compare response characteristics and determine whether the underlying model was the one advertised, a process known as model fingerprinting. Nearly half of the tested models failed. In other words, users may pay for a premium model but receive a downgraded version that performs almost 50% worse than the official service. **Your prompts pass through strangers.** US researchers tested relay services purchased from e-commerce platforms such as Taobao and Xianyu. Nine actively injected malicious code, several triggered decoy keys deliberately planted by the researchers to detect unauthorized use, and some sites even used stolen private keys to steal cryptocurrency. China’s Ministry of State Security has also publicly warned that some relay platforms intercept and steal user data, raising concerns about leaks of sensitive enterprise information. One statement is worth remembering: **As AI increasingly becomes infrastructure like water and electricity, we need to trust not only the company providing the AI service, but also the intermediary that determines where your questions ultimately go.** ## 3. Five Questions Enterprises Should Really Ask Combining the three risks above with the accountability and recordkeeping questions that every enterprise procurement team will ask gives us five questions for evaluating any AI intermediary layer: 1. **Where are my requests sent?** Do they go to official endpoints, or are they passed through another invisible intermediary? 2. **Which model am I using?** Is the model I specify the same one that actually runs? 3. **Who can see my prompts?** What happens while they are in transit, at rest, and within the intermediary itself? 4. **Who is accountable if something goes wrong?** Is there a legal entity, contract, or certification that can be held responsible? 5. **Are there records for reconciliation?** Can every call be reconstructed to show who made it, when it was made, which model was used, and how much it cost? Unofficial relays can answer almost none of these questions. Next, let’s see how MaiAgent answers them. ## 4. How MaiAgent AI Gateway Answers These Five Questions First, let’s explain what MaiAgent AI Gateway is. It is a gateway that lets enterprises manage all AI model calls centrally. When internal systems need to connect to language models, they no longer need to obtain their own keys or integrate separately with each provider. Everything passes through a single entry point, allowing the enterprise to centrally control which models are used, who can use them, usage levels, and spending. It is compatible with the OpenAI interface specification, so existing programs built with the official OpenAI library can connect simply by changing the endpoint to MaiAgent. This article uses three terms. An **endpoint** is the URL used to connect to a model service. **Bring your own key (BYOK)** means providing the platform with an API key your company obtained under its own contract with a provider so the platform can make calls on your behalf. A **fallback group** is a configuration that automatically switches to a backup model if the primary model encounters a problem. Architecturally, AI Gateway occupies the same position as a relay: between you and the model. The differences are all shown in the diagram below.

MaiAgent AI Gateway architecture and its five control dimensions — Figure 2: Where MaiAgent AI Gateway sits and its five control dimensions

### Question 1: Where Are Requests Sent? * **Amazon Bedrock connects directly through the official AWS library**. The code path contains no setting that could rewrite the endpoint. * **OpenAI and Anthropic use their official APIs**, while Google models use Google Cloud Vertex AI. * **Endpoints for platform-managed models are centrally configured by MaiAgent**. No account in your company can change them through the admin console or API, preventing accidental or malicious redirection to unofficial endpoints. Major providers use official endpoints. Some models are integrated through established commercial model marketplaces such as OpenRouter, and these are identified in the endpoint list. * **Your company configures endpoints for its custom models**, such as a self-hosted vLLM or OpenAI-compatible service connected through BYOK. Only your company can change these endpoints. * If your company needs it, we can provide the effective model list and connection endpoint for each model so they can be reconciled item by item. The accurate way to state this is: **The endpoint for every model your company uses can be inspected. Only we can change platform-managed endpoints, and only your company can change its custom endpoints.** ### Question 2: Which Model Is Used? * Organization administrators configure **allowlists and blocklists for available models** in the admin console. These are strictly enforced by the backend. If an assistant specifies a model outside the allowlist, the backend rejects the request outright. * The model name displayed externally by the Gateway (an alias) can be configured separately from the actual model. MaiAgent platform administrators can define which organizations may use each alias. * **Every request records the model that was actually resolved**, the provider, and whether BYOK was used. * Your company creates and orders its own fallback groups, which only organization owners can configure. The entire fallback chain is enabled only when the group identifier is sent as the model name in a call. Ordinary calls that do not specify a group are not automatically rerouted. Switching occurs only when the primary model times out, fails to connect, is rate-limited by the provider, or returns a server-side error. Request-side errors such as invalid parameters or invalid keys do not trigger a fallback, preventing configuration errors from being hidden. ### Question 3: Who Can See the Prompts? We do not avoid this question: **In platform-managed mode, MaiAgent can see them.** Conversation records are an inherent part of auditing and traceability. The difference lies in how they are protected and whether your company has a choice. The same three protections for data in transit and at rest apply at every trust level: encrypted connections across the site (HTTPS, with HSTS forcing browsers to use encrypted connections only), encryption at rest for databases and files, organization-level data isolation, and no code path that sends customer data for training. What differs among the three levels is whether the intermediary itself can see the prompts.

Trust levels for three model sources — Figure 3: Who can see the prompts for each of the three model sources

Trust Level	Who Can See the Prompts	Additional Protection
Platform-managed	MaiAgent and the model provider	Under its service terms, Amazon Bedrock does not use customer inputs to train models. Other providers are governed by their respective API service terms.
Bring your own key (BYOK)	MaiAgent and the provider contracted directly by your company	Your company’s keys are stored using industry-standard AES-256-GCM encryption. The ciphertext is intentionally excluded from queryable fields and cannot be retrieved through database queries. The admin console shows only a masked preview and does not allow the key to be read back.
On-premises deployment (both the platform and models are hosted in your company’s data center)	Only your company	Language models and embedding services (which convert documents into numerical representations for retrieval) can point to self-hosted services such as vLLM or Ollama on your company’s internal network. Reranking (selecting the most relevant passages) currently supports vLLM. When all three layers remain on the internal network, the data sent to models never leaves it, and even MaiAgent cannot see it.

{% hint style="warning" %} If only the model is hosted on your company’s internal network while the MaiAgent platform still uses the SaaS version, prompts still pass through the SaaS platform before being sent to the internal model. The data flow is the same as in the first row. {% endhint %} ### Question 4: Who Is Accountable If Something Goes Wrong? * MaiAgent is operated by Playma Co., Ltd., a company registered in Taiwan. It has a formal contractual relationship with your company and issues invoices in New Taiwan dollars. * The company holds ISO 27001:2022 information security management and ISO 27701:2019 privacy information management certifications, which are reviewed annually. * A disclosure list is available for the third-party services used by the platform (AWS, Cloudflare, Elastic Cloud, and Sentry) and their compliance certifications. You can request it as described in Section 5. We provide advance notice when adding critical services. * Once a fallback group is enabled, calls can switch to a backup model if the primary provider experiences an outage, rather than disappearing overnight like an account pool. ### Question 5: Are There Records for Reconciliation? Every request passing through the Gateway creates an audit record containing: * The request identifier, organization, API key, and member * The model alias requested by the client; the model and provider actually resolved; and whether BYOK was used * Input and output token counts, as well as token counts for cache creation and reads (temporary storage of repeated content) * Total latency, upstream latency, and time to first token * Processing and billing status The associated controls take effect **before** a call is made: **Organizations with insufficient credit are blocked before the call** based on wallet balance and the minimum balance threshold, with plan status checked at the same time. Reconciliation does not wait until after the call completes. Your company can review the usage dashboard and credit transaction details directly in the admin console, and every deduction includes the provider and model name. Returning to the opening question: the difference between MaiAgent and a relay is not where they sit, but whether your company can verify everything they do. ## 5. Getting Started 1. Contact to enable AI Gateway for your organization. 2. Select a trust level based on your company’s data classification: platform-managed, BYOK, or on-premises deployment. 3. Change the connection URL in your existing program to the MaiAgent Gateway endpoint and authenticate with your organization’s API key using Bearer authentication. You do not need to change the calling logic. 4. Configure available models, fallback groups, and usage limits in the admin console, then begin reconciling usage through the dashboard. {% hint style="info" %} References: The industry data and research findings about unofficial relays in this article are cited from the LINE TODAY program “TODAY 看世界” and its September 5, 2026 report, [“Use Top-Tier AI Without a VPN? Inside China’s Lucrative ‘AI Relay’ Industry!”](https://youtu.be/UtjsrVr2jZg). The report compiled information from Nanjing News, Yicai, National Business Daily, and German and US research teams. We did not independently verify the original research. {% endhint %} {% hint style="success" %} If your company’s security or audit team needs a complete platform security overview, a disclosure list of third-party services, or a technical meeting to review your requirements item by item, please contact your sales representative. We will provide the appropriate documentation. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.maiagent.ai/tech/en/platform-development/ai-gateway-security.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.