> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/tech/en/platform-development/gai-lan.md). # Overview Platform deployment overview: the MaiAgent platform itself does not require a GPU. Model services can use cloud API inference or self-hosted on-premises GPUs, with SaaS, private cloud, on-premises, an In addition to SaaS, MaiAgent offers self-hosted platform options (private cloud and on-premises). The MaiAgent platform itself requires only general-purpose computing resources and does not need GPU services. The model services used by MaiAgent (LLM, Embedding Model, Reranker Model) require computing power and can use either cloud API inference services or self-hosted on-premises GPUs.\\ You can also use a hybrid cloud architecture, with MaiAgent hosted locally (private cloud or on-premises) and model services (LLM, Embedding Model, Reranker Model) in the cloud. The three model types support different cloud providers (for example, Embedding and Reranker do not support Azure or Google Vertex AI). See [Cloud Model Inference API Services](/tech/en/platform-development/cloud-model-inference-api-service.md) for a comparison, and [On-Premises Model Service Integration](/tech/en/platform-development/onprem-model-services.md) for models that can run on on-premises hosts and how to connect them. If data security and on-premises cost considerations lead you to use on-premises computing resources in the future, you can switch to a self-hosted inference service (the Embedding model cannot be changed after a knowledge base is created; to use another Embedding model, create a new knowledge base and reimport the documents). ## MaiAgent Platform Overview **MaiAgent** is a complete generative AI platform that provides end-to-end services from the system backend to the user frontend. Its scalable microservices architecture can be built using managed cloud services (the AWS, GCP, and Azure columns below) or installed with a deployment package on cloud virtual machines or in a customer's data center. It supports three deployment types—single-host Docker, distributed Docker, and Kubernetes—for flexible deployment according to enterprise needs. The platform core is built on **Docker** and combines service modules covering APIs, task scheduling, data storage, cache management, and frontend and admin applications. Its overall design ensures **high availability, elastic scalability, and cross-cloud integration**, while addressing security and maintainability. | Service | Purpose | AWS | GCP | Azure | Virtual Machines, On-Premises (Deployment Package) | | ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | MaiAgent Server | MaiAgent core, APIs, and system admin console | ECS (Fargate) | Cloud Run | Container Apps | Django on Docker | | MaiAgent Worker Server | Workers for MaiAgent queued and asynchronous services, especially streaming message output | ECS (Fargate) | Cloud Run | Container Apps | Django on Docker | |

MaiAgent Worker Server
(Low-Priority)

| Workers for MaiAgent queued and asynchronous services, especially document vectorization | ECS (Fargate) | Cloud Run | Container Apps | Django on Docker | | Scheduler (Celery Beat) | Trigger periodic tasks; exactly one instance runs | ECS (Fargate) | Cloud Run | Container Apps | Celery Beat on Docker | | Message Broker | Deliver background tasks to workers for processing | Amazon MQ (RabbitMQ), or shared with the memory cache (ElastiCache Redis) | Shared with the memory cache (Memorystore Redis) | Shared with the memory cache (Azure Managed Redis), or RabbitMQ on Azure VM | RabbitMQ on Docker | | Reverse Proxy, Load Balancing | External HTTPS entry point and TLS termination | ALB | Cloud Load Balancing | Container Apps built-in ingress (API); frontends through Azure Front Door | Nginx (Traefik or Cilium Gateway for Kubernetes deployments) | | MaiAgent Admin Frontend | MaiAgent administration platform | S3 + CloudFront | GCS+Google Cloud CDN | Storage Account static website + Azure Front Door | Nginx + Static Files on Docker | | MaiAgent Web Chat Frontend | MaiAgent web chat frontend | S3 + CloudFront | GCS+Google Cloud CDN | Storage Account static website + Azure Front Door | Nginx + Static Files on Docker | | Relational Database (RDB) - PostgreSQL | Store MaiAgent data | RDS | Cloud SQL | Azure Database for PostgreSQL Flexible Server (built-in PgBouncer) | PostgreSQL on Docker, with a PgBouncer connection pool in front | | Vector Database (Vector DB) - Elasticsearch | Store vectors required for RAG and memory features (knowledge bases can also use [Bring Your Own Vector Database](/tech/en/platform-development/bring-your-own-vector-db.md)) | Elasticsearch | Elasticsearch | Elasticsearch on Azure VM | Elasticsearch on Docker (with Kibana) | | Storage | Store uploaded files and static documents | S3 | GCS | Azure Blob Storage | Garage on Docker (default for new installations; existing environments continue using MinIO) | | Memory Cache - Redis | API cache; also serves as the message broker in some cloud architectures | ElastiCache | Memorystore | Azure Managed Redis | Redis on Docker | * The **AWS, GCP, and Azure columns** describe architectures built with managed services from each cloud provider, deployed and operated by MaiAgent using Terraform. Combinations of managed services on other clouds are evaluated per project; contact your MaiAgent representative. You can also install the deployment package on cloud virtual machines (as in the rightmost column). * The **virtual machine and on-premises column** also includes a monitoring interface for background jobs (Flower). Optional deployment package components include local model services (vLLM, requires a GPU), offline speech recognition, real-time voice (LiveKit), a Code Interpreter sandbox (Docker deployments only), Ask AI (MCP Server), and monitoring (VictoriaMetrics, VictoriaLogs, Grafana). * See [Deployment Types and Feature Comparison](/tech/en/platform-development/onprem-deployment-types.md) and [System Requirements](/tech/en/platform-development/system-requirements.md) for deployment types and host specifications. ## Model Services The **MaiAgent** platform is designed to support both “**cloud API inference services**”與 and “**self-hosted GPU environments**”. * With **cloud API inference services**, MaiAgent connects directly to various LLM, Embedding, and Reranker APIs, allowing rapid scaling to handle dynamic traffic, easy experimentation, and quick launches. * In **self-hosted GPU mode**, MaiAgent connects to model services deployed locally or in private data centers, making full use of GPU resources for optimized inference while ensuring data privacy and compliance. Users can switch between flexibility and cost according to their needs, or even combine both approaches, making **MaiAgent** a unified inference and service management layer. |


| Cloud API Inference Services | Self-Hosted GPU | | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | MaiAgent Platform Compatibility |

AWS Bedrock
Google Vertex AI
Azure
OpenAI, Anthropic, Cohere
OpenAI-compatible APIs (such as OpenRouter, Groq, xAI)

|

Servers equipped with NVIDIA GPUs, such as:
HPE (Hewlett Packard Enterprise)
Advantech
Cisco
Dell

| | Model Capabilities |

Closed-source models: high
Open-source models: same as self-hosted GPU

| Depends on open-source model releases | | Speed | Depends on the model and provider | Depends on the model, GPU specifications, and concurrent request count | | Investment Cost |

Token API fees
(pay-as-you-go)

|

Hardware costs

Data center costs

Staff costs for hardware and model operations and maintenance

Hardware depreciation

| | Concurrent Requests | Depends on cloud service provider support | Depends on GPU specifications, quantity, and model; contact MaiAgent for an assessment | | Data Security | Use cloud service providers that commit to not using data for training (AWS, GCP, Azure) | High confidentiality and maximum security | | Personal Data Issues | Use a DLP Server or service to remove personal data | None | For cloud models available to your organization, see [Model Access Permissions](https://docs.maiagent.ai/org/model-access) in the user manual. For self-hosted GPUs, see [GPU Compute Hardware Planning](/tech/en/platform-development/gpu.md) for models that can run on the host and recommended GPUs, [System Requirements](/tech/en/platform-development/system-requirements.md) for host specifications, and [On-Premises Model Service Integration](/tech/en/platform-development/onprem-model-services.md) for connection methods. ## Platform Deployment Environments To meet customer needs during development, testing, and launch, our software platform offers multiple environment tiers. These environments ensure that every step from development to production is properly validated and controlled. ### Environment Architecture Our platform follows industry best practices and flexibly provides the following environments according to customer needs: | Environment | Notes | Primary Purpose | Characteristics | | ----------- | ---------------------------------------------------------------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | PROD | Within the bank | Production environment | Live external services using real data, with high stability and security requirements. | | UAT | Within the bank | User acceptance testing | For customer and business unit acceptance testing to confirm that system features meet requirements; similar to production. | | SIT | Within the bank, as integration with internal systems may be required | System integration testing | Verify integration and compatibility between modules and services using test data close to real data. | | DEV | Value-added features developed outside the bank to accelerate delivery | Development and testing environment | For code development and unit testing using simulated data; frequently updated, with tolerance for errors. | ### Environment Combinations Customers can choose an appropriate combination of environments according to project needs, for example: * **PROD only**: Suitable for small projects or straightforward launch needs, deploying directly to production. * **PROD + UAT**: Suitable for projects requiring acceptance testing, ensuring features meet requirements before launch. * **Full environment combination (DEV + SIT + UAT + PROD)**: Suitable for large or complex projects requiring a complete development, integration testing, and acceptance process. We configure environments flexibly according to customer needs to balance cost and quality. ### How Environments Are Isolated For private cloud and on-premises installations using the deployment package, **each environment is a separate installation**: each has its own deployment configuration (the environment identifier is dev, staging, uat, or prod; SIT in the table above usually corresponds to staging), installed on its own hosts or Kubernetes cluster. Databases, object storage, search indexes, keys, and certificates are not shared. There is no platform-level data synchronization between environments. To move settings configured in a test environment to production, use skill package export and import in the admin console for skills; Agent settings and knowledge base documents must be recreated and uploaded in production. * **Hosts**: Dedicated hosts or clusters are recommended for production. DEV and SIT can share a host but must be separate installations (with different deployment directories and ports or domains). * **Network**: Each environment has its own domain (or IP and port) and TLS certificate. When SIT integrates with internal systems, the firewall allows only that environment's hosts. * **Model services**: Environments can share the same model services (for example, one GPU host serving multiple environments) or each connect to cloud models. * **Accounts and permissions**: User accounts and roles are configured independently in each environment. Centralized management through SSO or LDAP identity synchronization is recommended. * **Promotion**: Upgrade and perform acceptance testing in UAT first, then schedule the production upgrade. See [Versions and Upgrades](/tech/en/platform-development/onprem-versioning-and-upgrades.md) for the upgrade process. MaiAgent operates the SaaS environment tiers; customers do not need to provision them. To separate testing and production use in SaaS, use different organizations. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.maiagent.ai/tech/en/platform-development/gai-lan.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.