Overview

In addition to SaaS services, the MaiAgent platform also offers self-hosted deployment options (private cloud and on-premises). The MaiAgent platform itself requires only general-purpose computing resources and does not require GPU services. The model services used by MaiAgent (LLM, Embedding Model, and Reranker Model) require computing power, which can be provided by either cloud API inference services or self-hosted on-premises GPUs.\

A hybrid cloud architecture is also supported, with MaiAgent deployed on-premises (in a private cloud or local environment) and the model services (LLM, Embedding Model, and Reranker Model) hosted in the cloud. MaiAgent supports LLM, Embedding model, and Reranker model services from all cloud service providers (CSPs). If data security requirements change or on-premises costs decrease in the future, you can immediately switch to on-premises computing resources.

MaiAgent Platform Overview

MaiAgent is a comprehensive generative AI platform that provides end-to-end services, from the system backend to the user frontend. It uses a scalable microservices architecture and supports major cloud environments (AWS, GCP, Azure, and Oracle) as well as self-hosted environments (Docker and K8s), enabling flexible deployment based on enterprise requirements.

Built on Docker, the platform combines a wide range of service modules covering APIs, task scheduling, data storage, cache management, and both administrative and user-facing applications. Its overall design ensures high availability, elastic scalability, and cross-cloud integration while maintaining security and operational manageability.

Service	Purpose	AWS	GCP	Azure	VMs
MaiAgent Server	MaiAgent core, APIs, and system administration backend	EKS(Fargate)	GKE	AKS	Django on Docker
MaiAgent Worker Server	MaiAgent worker for queued and asynchronous services, especially streaming message output	EKS(Fargate)	GKE	AKS	Django on Docker
MaiAgent Worker Server (Low-Priority)	MaiAgent worker for queued and asynchronous services, especially document vectorization	EKS(Fargate)	GKE	AKS	Django on Docker
MaiAgent Admin Frontend	MaiAgent administration platform	S3 + CloudFront	GCS+Google Cloud CDN	Azure Blob Storage + Azure CDN	Nginx + Static Files on Docker
MaiAgent Web Chat Frontend	MaiAgent web chat interface	S3 + CloudFront	GCS+Google Cloud CDN	Azure Blob Storage + Azure CDN	Nginx + Static Files on Docker
Relational Database (RDB) - PostgreSQL	Stores MaiAgent data	RDS	Cloud SQL	Azure Database	PostgreSQL on Docker
Vector Database (Vector DB) - Elasticsearch	Stores vectors required for RAG and memory features	Elasticsearch	Elasticsearch	Elasticsearch	Elasticsearch on Docker
Static Storage	Stores static files and web pages	S3	GCS	Azure Blob Storage	MinIO on Docker
Memory Cache - Redis	API cache and queues for queued and scheduled services	ElastiCache	Memorystore	Azure Cache	Redis on Docker

Model Services

The MaiAgent platform is designed to support both cloud API inference services and self-hosted GPU environments.

• With cloud API inference services, MaiAgent can connect directly to various LLM, Embedding, and Reranker APIs, scale rapidly, and handle dynamic traffic demand, making experimentation and rapid deployment easier.

• In self-hosted GPU mode, MaiAgent can connect to model services deployed locally or in a private data center, making full use of GPU resources and optimizing inference while ensuring data privacy and compliance.

Users can switch freely between the two options based on their flexibility and cost requirements, or even combine them, making MaiAgent a unified inference and service management layer.

	Cloud API Inference Services	Self-hosted GPU
MaiAgent Platform Compatibility	AWS Bedrock Google Vertex AI Azure AI Oracle OCI	HPE Advantech Cisco Dell
Model Capabilities	Closed-source models: High Open-source models: Same as self-hosted GPU	Depends on open-source model releases
Speed	Faster Claude 4 Sonnet: 80 token/s Gemini 2.5 Pro: 156 token/s	Moderate Llama3.3 70B - A100: 22.92 token/s - RTX PRO 6000 Blackwell: 73.54 token/s - H200: 145.40 token/s
Investment Cost	Token API fees (pay-as-you-go)	Hardware costs Data center costs Hardware and model operations personnel costs Hardware depreciation
Concurrent Requests	Depends on cloud service provider support	Depends on GPU specifications and number of GPUs - A100: 1.15 per card - RTX PRO 6000 Blackwell: 3.67 per card H200: 7.27 per card
Data Security	Uses cloud service providers (AWS, GCP, Azure, and Oracle) that commit not to using data for training	High confidentiality and maximum security
Personal Data Concerns	Use a DLP Server or service to remove personal data	None

Platform Deployment Environments

To meet different customer requirements throughout development, testing, and production rollout, our software platform provides multiple tiered environments. These environments ensure that every process, from development to production launch, is properly validated and controlled.

Environment Architecture

Our platform deployment follows industry best practices and flexibly provides the following environments based on customer requirements:

Environment	Notes	Primary Purpose	Characteristics
PROD	Internal	Production environment	Provides live external services, uses real data, and requires high stability and security.
UAT	Internal	User acceptance testing	Enables acceptance testing by customers and business units to confirm that system functionality meets requirements; closely resembles the production environment.
SIT	Internal, because integration with internal systems may be required	System integration testing	Validates integration and compatibility between modules and services using near-production test data.
DEV	Added features are developed externally to accelerate development	Development and testing environment	Enables developers to perform software development and unit testing using mock data; updated frequently and tolerant of errors.

Environment Combinations

Customers can select the appropriate combination of environments based on project requirements. For example:

• PROD only: Suitable for small projects or straightforward launch requirements, with direct deployment to the production environment.

• PROD + UAT: Suitable for projects that require acceptance testing, ensuring functionality meets requirements before launch.

• Complete environment combination (DEV + SIT + UAT + PROD): Suitable for large or complex projects that require a complete development, integration testing, and acceptance process.

We configure environments flexibly based on customer requirements to ensure the best balance between cost and quality.