> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/tech/en/quickstart/llm.md).

# Large Language Models (LLMs)

How to choose a large language model (LLM): select cloud or on-premises models based on your environment, data sensitivity, cost, and capabilities, learn which model families MaiAgent supports, and de

## **Key Selection Criteria**

Consider the following key factors when choosing a large language model:

1. **Environment**: determine whether to use cloud or on-premises models based on whether your environment can access the internet.
2. **Quality**: the model's ability to generate responses and follow instructions.
3. **Speed**: text generation speed and latency requirements to ensure adequate model response times.
4. **Pricing**: Credits pricing varies by model. Choose based on your usage needs and budget. See [Credits Billing](https://docs.maiagent.ai/others/credits) for pricing details.
5. **Other factors**: support for multimodal input and Function calling.

For comparisons of model quality, speed, and pricing, refer to continuously updated third-party evaluations such as [Artificial Analysis](https://artificialanalysis.ai/).

***

## Large Language Models Supported on MaiAgent

Models available on MaiAgent are continuously updated as providers release and retire models. This page lists providers and model families rather than specific model versions. For the models your organization can actually use, refer to the “[Model Access Permissions](https://docs.maiagent.ai/org/model-access)” page in the admin console. This page lists the models MaiAgent has made available to your organization, and organization owners can enable or disable models within that selection.

### Closed-Source Models

| Model Provider | Model Family     | Inference Services            |
| -------------- | ---------------- | ----------------------------- |
| OpenAI         | GPT, o series    | OpenAI, Azure, AWS Bedrock    |
| Anthropic      | Claude           | AWS Bedrock, Google Vertex AI |
| Google         | Gemini           | Google Vertex AI              |
| xAI            | Grok             | OpenAI-compatible API         |
| Alibaba        | Qwen (Max, Plus) | OpenAI-compatible API         |
| Amazon         | Nova             | AWS Bedrock                   |

### Open-Source Models

Open-weight models can be used on MaiAgent SaaS through cloud inference services or deployed on self-hosted GPUs in a private cloud or on premises. For hardware requirements, see the [GPU](/tech/en/platform-development/gpu.md) chapter.

| Model Provider | Model Family  | Inference Services on SaaS |
| -------------- | ------------- | -------------------------- |
| OpenAI         | GPT-OSS       | OpenAI-compatible API      |
| Google         | Gemma         | OpenAI-compatible API      |
| Alibaba        | Qwen, Qwen-VL | OpenAI-compatible API      |
| DeepSeek       | DeepSeek      | AWS Bedrock, Azure         |
| Mistral AI     | Mistral       | OpenAI-compatible API      |
| Moonshot AI    | Kimi          | OpenAI-compatible API      |
| IBM            | Granite       | OpenAI-compatible API      |

Models available for self-hosted on-premises deployment are determined by the model catalog in the MaiAgent deployment package, which currently includes families such as GPT-OSS, Gemma, Qwen, and Qwen-VL. Contact your MaiAgent representative for deployment planning.

{% hint style="info" %}
Each provider type supports a different range of LLM, Embedding, and Reranker models. See [Model API Inference Services](/tech/en/platform-development/cloud-model-inference-api-service.md) for details.
{% endhint %}

## Do Models Always Need Fine-Tuning?

As AI technology advances rapidly, language models have developed strong language understanding and generation capabilities and are widely used across many fields. For example, pretrained language models can readily handle everyday conversations, article generation, and simple Q\&A tasks.

However, when models face more challenging tasks in specialized fields such as medicine, law, or technical support, relying solely on pretrained models may not provide the best performance. Some developers choose **Fine-tuning**, which involves additional training for a specific domain to improve a model's specialized knowledge.

Fine-tuning is not the only solution. Two other effective approaches can achieve this goal: **Prompt Engineering** and **RAG (Retrieval-Augmented Generation)**.

### 1. Prompt Engineering: Optimize Model Performance with Precise Prompts

Prompt Engineering involves designing precise prompts to guide a model toward the desired results. The core approach is to formulate detailed instructions based on task requirements, helping the model narrow its response scope and understand the context and required output format.

Suppose a language model is intended to recommend suitable products based on a user's needs. A request such as “I want to buy a high-performance phone” may lead to an imprecise answer because “high-performance” can mean many things, such as processing speed, camera performance, or battery life.

To improve recommendation accuracy, we can use **Prompt Engineering** to design more specific questions or provide additional context that helps the model understand the user's needs. For example, use the following prompt instead:

> "I need a phone with **long battery life** and an **efficient processor**, priced between **USD 500 and USD 800**. Please recommend a few phones that meet these criteria."

### 2. RAG: Improve Generation with External Knowledge

**RAG** improves model performance by combining external knowledge retrieval with the generation process.

In traditional generation tasks, models rely solely on knowledge learned during pretraining. RAG uses a retrieval system to obtain external information in real time and combines it with the generative model to answer questions or generate text more accurately.

For example, in medicine, when a model is asked about a rare disease, RAG can first retrieve relevant information from a specialized medical database and then use it to generate a more accurate answer. This approach allows the model to produce high-quality responses by retrieving existing knowledge even if it has never encountered that information during training. It is particularly useful when knowledge needs to be updated in real time and can greatly expand the model's knowledge coverage.

{% hint style="success" %}
For a more detailed introduction to RAG, see the next chapter, [“RAG Knowledge Base Retrieval System Overview”](/tech/en/quickstart/rag.md).
{% endhint %}

Fine-tuning, Prompt Engineering, and RAG each have their own advantages and areas of application. Choose the most suitable strategy for your use case and requirements rather than relying solely on Fine-tuning.

**Prompt Engineering** offers a low-cost, flexible solution by designing precise prompts to guide models toward high-quality results.

**RAG** combines external knowledge with generation capabilities to provide more accurate answers when knowledge must be retrieved dynamically.

**Fine-tuning** can significantly improve model performance in specific domains, but requires substantial specialized data and additional resources. It can be considered a last resort when both **Prompt Engineering** and **RAG** fail to meet the need.

{% hint style="info" %}
**The MaiAgent platform does not provide model fine-tuning**: the admin console has no interface for training or fine-tuning models, and the platform does not use your data to train any models. The Fine-tuning discussed above is performed outside the platform by model providers or self-hosted model teams. Models that have already been fine-tuned can be connected to the platform as follows: connect fine-tuned models hosted by cloud providers using your own API key (see [AI Gateway: OpenAI-Compatible API Integration](/tech/en/ai-gateway/ai-gateway-openai-compatible.md)); deploy your own trained open-weight models as OpenAI-compatible model services in a private cloud or on premises and then connect them. See [On-Premises Model Service Integration](/tech/en/platform-development/onprem-model-services.md#external-openai-compatible). To give an Agent specialized knowledge, start with [System Prompts](/tech/en/advanced-genai-tech/system-prompt.md) and a [RAG Knowledge Base](/tech/en/quickstart/rag.md).
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.maiagent.ai/tech/en/quickstart/llm.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.