> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/tech/en/platform-development/gai-lan.md).
# Overview
In addition to SaaS services, the MaiAgent platform also offers self-hosted deployment options (private cloud and on-premises). The MaiAgent platform itself requires only general-purpose computing resources and does not require GPU services. The model services used by MaiAgent (LLM, Embedding Model, and Reranker Model) require computing power, which can be provided by either cloud API inference services or self-hosted on-premises GPUs.\\
A hybrid cloud architecture is also supported, with MaiAgent deployed on-premises (in a private cloud or local environment) and the model services (LLM, Embedding Model, and Reranker Model) hosted in the cloud. MaiAgent supports LLM, Embedding model, and Reranker model services from all cloud service providers (CSPs). If data security requirements change or on-premises costs decrease in the future, you can immediately switch to on-premises computing resources.
## MaiAgent Platform Overview
**MaiAgent** is a comprehensive generative AI platform that provides end-to-end services, from the system backend to the user frontend. It uses a scalable microservices architecture and supports major cloud environments (AWS, GCP, Azure, and Oracle) as well as self-hosted environments (Docker and K8s), enabling flexible deployment based on enterprise requirements.
Built on **Docker**, the platform combines a wide range of service modules covering APIs, task scheduling, data storage, cache management, and both administrative and user-facing applications. Its overall design ensures **high availability, elastic scalability, and cross-cloud integration** while maintaining security and operational manageability.
| Service | Purpose | AWS | GCP | Azure | VMs |
| ----------------------------------------------- | ----------------------------------------------------------------------------------------- | --------------- | -------------------- | ------------------------------ | ------------------------------ |
| MaiAgent Server | MaiAgent core, APIs, and system administration backend | EKS(Fargate) | GKE | AKS | Django on Docker |
| MaiAgent Worker Server | MaiAgent worker for queued and asynchronous services, especially streaming message output | EKS(Fargate) | GKE | AKS | Django on Docker |
| MaiAgent Worker Server
(Low-Priority)
| MaiAgent worker for queued and asynchronous services, especially document vectorization | EKS(Fargate) | GKE | AKS | Django on Docker |
| MaiAgent Admin Frontend | MaiAgent administration platform | S3 + CloudFront | GCS+Google Cloud CDN | Azure Blob Storage + Azure CDN | Nginx + Static Files on Docker |
| MaiAgent Web Chat Frontend | MaiAgent web chat interface | S3 + CloudFront | GCS+Google Cloud CDN | Azure Blob Storage + Azure CDN | Nginx + Static Files on Docker |
| Relational Database (RDB) - PostgreSQL | Stores MaiAgent data | RDS | Cloud SQL | Azure Database | PostgreSQL on Docker |
| Vector Database (Vector DB) - Elasticsearch | Stores vectors required for RAG and memory features | Elasticsearch | Elasticsearch | Elasticsearch | Elasticsearch on Docker |
| Static Storage | Stores static files and web pages | S3 | GCS | Azure Blob Storage | MinIO on Docker |
| Memory Cache - Redis | API cache and queues for queued and scheduled services | ElastiCache | Memorystore | Azure Cache | Redis on Docker |
## Model Services
The **MaiAgent** platform is designed to support both **cloud API inference services** and **self-hosted GPU environments**.
* With **cloud API inference services**, MaiAgent can connect directly to various LLM, Embedding, and Reranker APIs, scale rapidly, and handle dynamic traffic demand, making experimentation and rapid deployment easier.
* In **self-hosted GPU mode**, MaiAgent can connect to model services deployed locally or in a private data center, making full use of GPU resources and optimizing inference while ensuring data privacy and compliance.
Users can switch freely between the two options based on their flexibility and cost requirements, or even combine them, making **MaiAgent** a unified inference and service management layer.
|
| Cloud API Inference Services | Self-hosted GPU |
| ------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| MaiAgent Platform Compatibility | AWS Bedrock
Google Vertex AI
Azure AI
Oracle OCI
| HPE
Advantech
Cisco
Dell
|
| Model Capabilities | Closed-source models: High
Open-source models: Same as self-hosted GPU
| Depends on open-source model releases |
| Speed | Faster
Claude 4 Sonnet: 80 token/s
Gemini 2.5 Pro: 156 token/s
| Moderate
Llama3.3 70B
- A100: 22.92 token/s
- RTX PRO 6000 Blackwell: 73.54 token/s
- H200: 145.40 token/s
|
| Investment Cost | Token API fees
(pay-as-you-go)
| Hardware costs
Data center costs
Hardware and model operations personnel costs
Hardware depreciation
|
| Concurrent Requests | Depends on cloud service provider support | Depends on GPU specifications and number of GPUs
- A100: 1.15 per card
- RTX PRO 6000 Blackwell: 3.67 per card
H200: 7.27 per card
|
| Data Security | Uses cloud service providers (AWS, GCP, Azure, and Oracle) that commit not to using data for training | High confidentiality and maximum security |
| Personal Data Concerns | Use a DLP Server or service to remove personal data | None |
## Platform Deployment Environments
To meet different customer requirements throughout development, testing, and production rollout, our software platform provides multiple tiered environments. These environments ensure that every process, from development to production launch, is properly validated and controlled.
### Environment Architecture
Our platform deployment follows industry best practices and flexibly provides the following environments based on customer requirements:
| Environment | Notes | Primary Purpose | Characteristics |
| ----------- | ------------------------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PROD | Internal | Production environment | Provides live external services, uses real data, and requires high stability and security. |
| UAT | Internal | User acceptance testing | Enables acceptance testing by customers and business units to confirm that system functionality meets requirements; closely resembles the production environment. |
| SIT | Internal, because integration with internal systems may be required | System integration testing | Validates integration and compatibility between modules and services using near-production test data. |
| DEV | Added features are developed externally to accelerate development | Development and testing environment | Enables developers to perform software development and unit testing using mock data; updated frequently and tolerant of errors. |
### Environment Combinations
Customers can select the appropriate combination of environments based on project requirements. For example:
* **PROD only**: Suitable for small projects or straightforward launch requirements, with direct deployment to the production environment.
* **PROD + UAT**: Suitable for projects that require acceptance testing, ensuring functionality meets requirements before launch.
* **Complete environment combination (DEV + SIT + UAT + PROD)**: Suitable for large or complex projects that require a complete development, integration testing, and acceptance process.
We configure environments flexibly based on customer requirements to ensure the best balance between cost and quality.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.maiagent.ai/tech/en/platform-development/gai-lan.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.