> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/agent-ops/evaluation-insights.md).
# Evaluation Insights Report
This guide explains how to use the Evaluation Insights feature to quickly understand your AI assistant's test results and receive system-generated improvement recommendations.
## What Are Evaluation Insights?
Evaluation Insights is an extension of AgentOps automated testing. After completing a batch test, the system not only displays metric scores but also automatically analyzes test results and generates an "Insights Report."
This report tells you:
* **How the test results performed overall**
* **Where the main issues are**
* **How to make specific improvements**
* **Which items to prioritize**
Think of it as having a senior consultant review your test report and provide you with an improvement recommendation document.
## Core Advantages of Evaluation Insights
### 📊 Automated Analysis
No need to manually interpret complex scoring data — the system automatically identifies problem patterns and compiles them into an easy-to-understand report.
### 🎯 Priority Recommendations
The system recommends which items to address first based on issue severity and scope of impact, making your improvement efforts more efficient.
### 💡 Specific Improvement Directions
It doesn't just tell you "there's a problem" — it also provides concrete improvement suggestions, such as which settings to adjust and what content to supplement.
### 🌐 Multi-Language Support
Insights reports support multiple languages including Traditional Chinese, Simplified Chinese, and English. You can choose the language that best suits your team's needs.
## How to Use the Evaluation Insights Feature
### Step 1: Complete a Batch Test
1. Go to "AgentOps" → "Automated Testing"
2. Run a batch test (refer to [AgentOps User Guide](https://github.com/Playma-Co-Ltd/maiagent-user-guide-gitbook/blob/main/en/agent-builder/agentops-automated-test.md))
3. Wait for the test to complete
### Step 2: View Evaluation Results
1. Go to the test result detail page
2. Review each scoring metric
3. Find the "Insights Report" section
### Step 3: Select the Report Language
In the Insights Report section, you can choose the display language for the report:
* **Traditional Chinese**: Suitable for Taiwan-based teams
* **Simplified Chinese**: Suitable for Mainland China-based teams
* **English**: Suitable for international teams or situations requiring English reports
After selecting a language, the system automatically generates the insights report in the corresponding language.
### Step 4: Read the Insights Report
The insights report typically includes the following content:
#### 1. Overall Evaluation Summary
* Test pass rate
* Key strengths
* Areas needing improvement
#### 2. Issue Analysis
* Answer relevance issues
* Safety risks (bias, offensiveness, hallucinations)
* Response speed issues
#### 3. Improvement Suggestions
* Priority items (labeled as "High Priority," "Medium Priority," or "Low Priority")
* Specific improvement steps
* Expected improvement outcomes
### Step 5: Make Improvements Based on Suggestions
1. Address issues in order of the priority listed in the report
2. Adjust AI assistant settings based on the specific improvement steps
3. After making adjustments, run the test again to verify the results
## How Evaluation Insights Are Calculated
The system considers the following factors to generate the insights report:
* **Scoring Metrics**: The levels and distribution of each score
* **Failed Cases**: The number and reasons for failed cases
* **Problem Patterns**: The frequency of recurring similar issues
* **Best Practices**: Industry standards and optimization recommendations
{% hint style="info" %}
Generating the insights report requires some processing time, typically completed within 1-2 minutes after the test finishes.
{% endhint %}
## Practical Use Cases
### 🏥 Medical Clinic Customer Service
**Test Results**:
* Pass rate: 72%
* Hallucination score: High
**Insights Report Suggestion**:
> "The test found that some responses contained unverified medical advice. It is recommended to explicitly restrict the role instructions: 'Do not provide diagnostic or treatment advice; only provide appointment scheduling and clinic information.'"
### 🏢 Enterprise Internal Knowledge Management
**Test Results**:
* Pass rate: 88%
* Answer relevance: Low
**Insights Report Suggestion**:
> "The test found that some responses were too brief or did not fully answer the question. It is recommended to add more detailed operating steps and examples to the knowledge base."
## FAQ
### Q1: Can the insights report be downloaded?
Currently, the insights report can only be viewed within the system. You can copy the report content and compile it into an internal document.
### Q2: Does the insights report update automatically with new test results?
Yes, each time a new batch test is run, the system generates a new insights report reflecting the latest test results.
### Q3: Can I switch the language of the insights report?
Yes, you can switch the report language at any time, and the system will regenerate the report in the selected language.
### Q4: Do I have to implement all the suggestions in the insights report?
Not necessarily. You can prioritize "High Priority" items based on actual needs and available resources. The system's suggestions are reference directions — you can adjust them according to your business requirements.
### Q5: Why didn't my test generate an insights report?
Possible reasons:
* Too few test cases (at least 10 are recommended)
* The test has not finished yet
* The system is still processing (please wait 1-2 minutes)
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/agent-ops/evaluation-insights.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.