# Overview In addition to providing SaaS services, the MaiAgent platform also offers self-hosted solutions (private cloud, on-premises). The MaiAgent platform itself only requires general computing resources without GPU services. The model services used by MaiAgent (LLM, Embedding Model, Reranker Model) require computing power, which can utilize either cloud API inference services or on-premises GPU infrastructure. A hybrid cloud architecture is also possible, with MaiAgent on-premises (private cloud, on-premises) and model services (LLM, Embedding Model, Reranker Model) in the cloud. MaiAgent supports LLM, Embedding model, and Reranker model provided by all cloud service providers (CSPs). If data security concerns arise or on-premises costs decrease in the future, switching to on-premises computing power can be done immediately. ## MaiAgent Platform Overview **MaiAgent** is a comprehensive generative AI platform providing end-to-end services from system backend to user frontend. The platform adopts a scalable microservices architecture and supports mainstream cloud environments (AWS, GCP, Azure, Oracle) as well as on-premises environments (Docker, K8s), allowing flexible deployment based on enterprise needs. The platform core is built on **Docker**, combining diverse service modules covering API, task scheduling, data storage, cache management, and frontend/backend applications. Its overall design ensures **high availability, scalability, and cross-cloud integration capabilities** while maintaining security and operability. | Service Item | Purpose | AWS | GCP | Azure | VMs | | ------------------------------------------- | --------------------------------------------------------------------------------------------- | --------------- | -------------------- | ------------------------------ | ------------------------------ | | MaiAgent Server | MaiAgent core, API, system management backend | EKS(Fargate) | GKE | AKS | Django on Docker | | MaiAgent Worker Server | MaiAgent queue processing, asynchronous service worker, especially for message stream output | EKS(Fargate) | GKE | AKS | Django on Docker | | MaiAgent Worker Server (Low-Priority) | MaiAgent queue processing, asynchronous service worker, especially for document vectorization | EKS(Fargate) | GKE | AKS | Django on Docker | | MaiAgent Admin Frontend | MaiAgent management platform | S3 + CloudFront | GCS+Google Cloud CDN | Azure Blob Storage + Azure CDN | Nginx + Static Files on Docker | | MaiAgent Web Chat Frontend | MaiAgent web chat frontend | S3 + CloudFront | GCS+Google Cloud CDN | Azure Blob Storage + Azure CDN | Nginx + Static Files on Docker | | Relational Database (RDB) - PostgreSQL | Store MaiAgent data | RDS | Cloud SQL | Azure Database | PostgreSQL on Docker | | Vector Database (Vector DB) - Elasticsearch | Store vectors needed for RAG and memory functions | Elasticsearch | Elasticsearch | Elasticsearch | Elasticsearch on Docker | | Static Storage | Store static files and web pages | S3 | GCS | Azure Blob Storage | MinIO on Docker | | Memory Cache - Redis | API cache, queue scheduling service queue | ElastiCache | Memorystore | Azure Cache | Redis on Docker | ## Model Services The **MaiAgent** platform is designed to be compatible with both **cloud API inference services** and **self-hosted GPU environments**. * With **cloud API inference services**, MaiAgent can directly integrate with various LLM, Embedding, and Reranker APIs, enabling rapid scaling and support for dynamic traffic demands, facilitating experimentation and quick deployment. * In **self-hosted GPU mode**, MaiAgent can integrate with models deployed locally or in private data centers, fully utilizing GPU resources and optimizing inference while ensuring data privacy and compliance. Users can freely switch between flexibility and cost based on their needs, or even use a hybrid approach, making **MaiAgent** a unified inference and service management layer. | | Cloud API Inference Services | Self-hosted GPU | | ------------------------------- | ------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | MaiAgent Platform Compatibility |

AWS Bedrock
Google Vertex AI
Azure AI
Oracle OCI

|

HPE
Advantech
Cisco
Dell

| | Model Capabilities |

Closed-source models: High
Open-source models: Same as self-hosted GPU

| Depends on open-source model releases | | Speed |

Faster
Claude 4 Sonnet: 80 token/s
Gemini 2.5 Pro: 156 token/s

|

Medium
Llama3.3 70B: 25.01 token/s (H100 example)

| | Investment Cost |

Token API fees
(pay-as-you-go)

|

Hardware costs
Data center costs
Hardware and model maintenance personnel costs
Hardware depreciation

| | Concurrent Users | Based on cloud service provider support |

Based on GPU quantity
(Currently about 25 users per H100 GPU)

| | Data Security | Using cloud service providers (AWS, GCP, Azure, Oracle) that promise not to use data for training | Highest confidentiality, most secure | | Personal Data Issues | Use DLP Server or services to remove personal data | None | ## Platform Deployment Environments To meet different customer needs during development, testing, and deployment processes, our software platform provides multiple layered environments. These environment setups ensure proper validation and control of every process from development to production. ### Environment Architecture Our platform setup follows industry best practices and flexibly provides the following environments based on customer needs: | Environment Name | Notes | Main Purpose | Characteristics | | ---------------- | --------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | PROD | Internal | Production Environment | Official external service, uses real data, requires high stability and security | | UAT | Internal | User Acceptance Testing | For customer and business unit acceptance, verifying system functionality meets requirements, environment similar to production | | SIT | Internal, may need to connect to internal systems | System Integration Testing | Validates integration and compatibility between different modules and services, uses near-real test data | | DEV | External development for added features to speed up | Development Testing Environment | For developer program development and unit testing, uses simulated data, frequent updates, tolerates errors | ### Environment Combinations Different customers can choose suitable environment combinations based on project requirements, such as: * **PROD Only**: Suitable for small projects or simple deployment needs, directly deployed to production environment. * **PROD + UAT**: Suitable for projects requiring acceptance testing, ensuring functionality meets requirements before going live. * **Complete Environment Combination (DEV + SIT + UAT + PROD)**: Suitable for large or complex projects requiring complete development, integration testing, and acceptance processes. We flexibly configure based on customer needs, ensuring optimal balance between cost and quality.