> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/application/optimizer-assistant.md).

# AI Assistant Optimization Consultant

## Overview

When an AI assistant produces inaccurate answers, formatting errors, hallucinated information, or other issues, it typically requires manual review of conversation records to locate problems. The **Optimization Assistant** can automate this diagnostic process:

1. Automatically retrieve conversation records, AI responses, and knowledge base retrieval results
2. Determine whether the issue is a **Prompt Problem** (AI received the right data but answered incorrectly) or a **Knowledge Base Document Problem** (AI didn't receive the right data)
3. Generate specific modification recommendations with Before / After comparisons

### Components

| Component             | Description                                                                                                           |
| --------------------- | --------------------------------------------------------------------------------------------------------------------- |
| **AI Assistant**      | Serves as the carrier for the optimization consultant, using Agent mode                                               |
| **Role Instructions** | Defines the diagnostic process, analysis principles, and output format                                                |
| **MCP Tool**          | Allows the optimization assistant to call the MaiAgent API to query conversation records and assistant configurations |
| **Knowledge Base**    | Bound to the knowledge base used by the target assistant, for verifying RAG retrieval results                         |

***

## Prerequisites

| Item                              | Description                                         | How to Obtain                                   |
| --------------------------------- | --------------------------------------------------- | ----------------------------------------------- |
| Assistant to Analyze              | The target AI assistant you want to optimize        | Admin Console → AI Assistant List               |
| Target Assistant's Knowledge Base | The knowledge base attached to the target assistant | Click the target assistant → Knowledge Base tab |
| API Key                           | For MCP tool authentication                         | Profile → API Keys (see instructions below)     |

***

## Step 1: Create an AI Assistant

1. Left sidebar → <mark style="color:blue;">AI Assistants</mark> → Click <mark style="color:blue;">Create AI Assistant</mark>
2. Fill in basic settings:

| Field         | Recommended Value                                                              |
| ------------- | ------------------------------------------------------------------------------ |
| **Name**      | `Optimization Assistant` or `AI Assistant Optimization Consultant`             |
| **LLM Model** | Claude Sonnet 4.6 or above (requires strong analytical reasoning capabilities) |

3. Switch to the <mark style="color:blue;">Response Mode Settings</mark> tab, select <mark style="color:blue;">Agent Mode</mark>

{% hint style="danger" %}
**Response mode must be set to Agent Mode**. RAG Q\&A mode cannot use MCP tools, which will prevent the optimization assistant from querying conversation records and assistant configurations.
{% endhint %}

4. Click <mark style="color:blue;">Save</mark>

***

## Step 2: Set Up Role Instructions

1. In the same page's <mark style="color:blue;">Response Mode Settings</mark> tab, find the **Role Instructions** section below
2. Paste the following standard role instructions in full

{% hint style="info" %}
The role instructions are the core of the optimization assistant, defining the diagnostic process, seven optimization principles, and output format. It is recommended to use the standard version as-is without modification.
{% endhint %}

<details>

<summary>Click to expand: Standard Role Instructions</summary>

```
# Role Definition

You are the "AI Assistant Optimization Consultant", responsible for diagnosing the response quality of AI assistants on the MaiAgent platform and providing adjustment recommendations for prompts or knowledge base documents.

Your technical foundation is RAG (Retrieval-Augmented Generation): the AI assistant retrieves relevant content from the knowledge base and then answers users based on the retrieval results. Therefore, problems only come from two sources:
1. **Prompt Problem** — AI received the right data but answered incorrectly
2. **Knowledge Base Document Problem** — AI didn't receive the right data

---

# Diagnostic Process

## Step 1: Collect Information

After the user provides a conversation_id, execute in order:

1. Use the MaiAgent API tool to query the message list with conversation_id and find the recordId
2. Use the recordId to get conversation record details (including citationNodes, context, scores)
3. Get the analyzed assistant's Prompt (instructions + skills) from the chatbot.id in the record
4. If there are skills, get each skill's instructions individually

**All information must be obtained before proceeding to diagnosis.**

## Step 2: Display Diagnostic Summary

**Analyzed Assistant**: [chatbot.name] (Model: [llm.name])
**User Question**: [inputMessage]
**AI Response Summary**: [First 200 characters of outputMessage]
**Retrieved Sources**: [fileName list from citationNodes]
**Scores**: Faithfulness [X] / Answer Relevance [X] / Reference Relevance [X]
**Diagnosis Result**: [Prompt Problem / Knowledge Base Document Problem / Knowledge Base Gap]

## Step 3: Problem Classification

**Q1: Do the citationNodes contain the correct answer?**
- Yes → **Prompt Problem**
- No → Continue to Q2

**Q2: Does the knowledge base contain the correct answer?**
- Yes, but AI didn't retrieve it → **Knowledge Base Document Problem**
- No → Inform the user that **new knowledge base content needs to be added**

---

# Seven Optimization Principles

| # | Principle | Core Concept |
|---|-----|---------|
| 1 | Restrict Knowledge Source | "Only use... must not use..." |
| 2 | Mandatory Citations | "Must cite sources" |
| 3 | Authorize Refusal | "You can say you don't know" + standard phrasing |
| 4 | Clear Process | "Step 1... Step 2..." |
| 5 | Absolute Boundaries | "Absolutely prohibited" "Strictly forbidden" |
| 6 | Fallback Mechanism | "In the following cases, must transfer to human agent" |
| 7 | Structured Format | XML tags or clear delimiters |
```

</details>

3. Click <mark style="color:blue;">Save</mark>

***

## Step 3: Create MCP Tool

The MCP tool allows the optimization assistant to query the MaiAgent API to retrieve conversation records and assistant configurations.

1. Left sidebar → <mark style="color:blue;">Tools</mark> → Click <mark style="color:blue;">Add Tool</mark> in the upper right corner
2. Switch the <mark style="color:blue;">Tool Type</mark> from API to <mark style="color:blue;">MCP</mark>
3. Fill in the following settings:

| Field            | Value                         |
| ---------------- | ----------------------------- |
| **Display Name** | `MaiAgent API`                |
| **Description**  | `AI Response Analysis Tool`   |
| **MCP Tool URL** | `https://mcp.maiagent.ai/mcp` |

4. After creation, go to the assistant settings → <mark style="color:blue;">Tools</mark> tab to see the bound MCP tool
5. **Authentication Headers (MCP Headers)** — Enter the following JSON:

```json
{
  "Authorization": "Api-Key {Your API Key}"
}
```

{% hint style="info" %}
**How to obtain an API Key?**

Click the avatar in the upper right corner → <mark style="color:blue;">Profile</mark> → Switch to the <mark style="color:blue;">API Keys</mark> tab → Click <mark style="color:blue;">Create New Key</mark>, and copy the generated API Key.
{% endhint %}

{% hint style="danger" %}
The API Key is only displayed once after creation. Copy and store it securely immediately.
{% endhint %}

4. **Allowed Tools** — Check the following 5:

| Tool                  | Purpose                          |
| --------------------- | -------------------------------- |
| `debug_auth`          | Verify authentication is correct |
| `list_api_categories` | List API categories              |
| `search_apis`         | Search API endpoints             |
| `get_api_details`     | Get API detailed specifications  |
| `call_api`            | Execute API calls                |

5. **Tool Prompt** — Paste the following content in full:

<details>

<summary>Click to expand: Standard Tool Prompt</summary>

```
MaiAgent platform API calling tool. Used to query conversation records, retrieve assistant configurations, and read knowledge base content.

## Core Operation Flow

### Step 1: Find recordId from conversation_id
call_api(operation_id: 'v1MessagesList', query_params: { conversation: '<conversation_id>', pageSize: 50 })

### Step 2: Get conversation record details
call_api(operation_id: 'v1RecordsRetrieve', path_params: { id: '<record_id>' })

### Step 3: Get the analyzed assistant's Prompt
call_api(operation_id: 'v1ChatbotsRetrieve', path_params: { id: '<chatbot_id>' })

### Step 4: Get Skill instructions (if skills exist)
call_api(operation_id: 'v1SkillsRetrieve', path_params: { id: '<skill_id>' })

### Step 5 (Optional): Get TextNode details
call_api(operation_id: 'v1ChatbotTextNodesRetrieve', path_params: { id: '<textnode_id>' })

Steps 1-4 must be executed sequentially; each step's output is the next step's input.
```

</details>

6. Click <mark style="color:blue;">Save</mark>

***

## Step 4: Bind Knowledge Base

The optimization assistant needs to bind the **knowledge base used by the target assistant** to verify RAG retrieval results during diagnosis.

1. Switch to the <mark style="color:blue;">Knowledge Base Settings</mark> tab
2. Click <mark style="color:blue;">Add Knowledge Base</mark> and check all knowledge bases used by the target assistant
3. Click <mark style="color:blue;">Save</mark>

{% hint style="info" %}
If you need to analyze multiple AI assistants, bind all their knowledge bases to the optimization assistant. It is recommended to set the retrieval chunks count to **12** in "Advanced Settings".
{% endhint %}

***

## Step 5: Verify Setup

### Verify MCP Tool Connection

In the optimization assistant's chat box, enter:

```
Please verify if the MCP tool connection is working properly
```

The optimization assistant should successfully call `debug_auth` and return authentication information.

### Verify Diagnostic Functionality

Obtain a conversation\_id from the target assistant's conversation records:

1. Go to the <mark style="color:blue;">All Conversations</mark> page
2. Click any conversation; the ID in the URL is the conversation\_id

In the optimization assistant, enter:

```
Please analyze this conversation: {conversation_id}
```

The optimization assistant will automatically execute the following steps in order:

1. Search for available API tools (`search_apis`)
2. Call `v1MessagesList` to query conversation messages
3. Call `v1RecordsRetrieve` to get conversation record details
4. Call `v1ChatbotsRetrieve` to get the analyzed assistant's Prompt
5. Produce a complete diagnostic summary and modification recommendations

***

## Usage Guide

### Analyze by conversation\_id

```
Please analyze this conversation: abc123-def456-ghi789
```

The optimization assistant will automatically retrieve conversation records, AI responses, knowledge base retrieval results, and assistant prompts to produce a complete diagnostic report.

### Analyze by Pasting Text Directly

If you don't have a conversation\_id, you can provide the information directly:

```
Please help me analyze the following issue:

[User Question] How do I apply for credit card loss reporting?
[AI Response] (paste the AI's actual response)
[Problem Description] The AI mentioned a non-existent "online loss reporting" feature
```

### Interpreting Diagnostic Results

| Diagnosis Type                      | Meaning                                                       | Next Steps                                             |
| ----------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------ |
| **Prompt Problem**                  | AI received the right data but the response approach is wrong | Modify role instructions or Skills per recommendations |
| **Knowledge Base Document Problem** | Document structure caused AI to retrieve incorrect content    | Rewrite knowledge base documents per recommendations   |
| **Knowledge Base Gap**              | Knowledge base simply doesn't have relevant content           | New knowledge base documents need to be added          |

***

## FAQ

{% hint style="info" %}
**Q: What should I do if the MCP tool connection fails?**

1. Confirm the response mode is **Agent Mode** (most common cause)
2. Confirm the API Key is valid and not expired
3. Confirm the MCP URL is `https://mcp.maiagent.ai/mcp`
4. Confirm Headers use the `Api-Key` format (e.g., `Api-Key xxxxxxxx.xxxxxxxx`)

**Q: Can one optimization assistant analyze multiple assistants?** Yes. As long as the API Key belongs to the same organization and all target assistants' knowledge bases are bound.

**Q: Do I need to create one for each client?** Yes. Each client organization needs its own optimization assistant, since API Keys and knowledge bases are organization-specific.
{% endhint %}

***

## Setup Checklist

* [ ] Create AI Assistant (Name: Optimization Assistant)
* [ ] Select model (Claude Sonnet 4.6 or above)
* [ ] **Set to Agent Mode**
* [ ] Paste standard role instructions
* [ ] Create MCP tool
  * [ ] MCP URL: `https://mcp.maiagent.ai/mcp`
  * [ ] Headers: API Key
  * [ ] Check 5 Allowed Tools
  * [ ] Paste standard Tool Prompt
* [ ] Bind target assistant's knowledge base
* [ ] Verify MCP connection
* [ ] Verify diagnostic functionality


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/application/optimizer-assistant.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.