> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/maiagent-user-guide/en/serve/daily-operations/reply/inbox-analysis.md).

# Conversation Platform Analytics

Use AI to automatically analyze topic clusters, resolution rates, and improvement suggestions for conversations on a single channel

Inbox Analysis uses AI to automatically identify topics, assess resolution status, and produce specific improvement suggestions from real conversations on a particular channel within a specified date range, helping you continuously improve your Agent's conversation quality.

## What Is Inbox Analysis? <a href="#what-is-inbox-analysis" id="what-is-inbox-analysis"></a>

Once customer service conversations reach a certain volume, totals and satisfaction scores alone are not enough. You also want to know:

* **What are users actually asking?** Which topics come up most often?
* **How many conversations were actually resolved?** Which types of questions have low resolution rates?
* **Why were issues left unresolved?** Is content missing from the knowledge base, are instructions too restrictive, or did a tool fail?
* **How are response times and costs distributed?** Which topics take longer or use more tokens?

Inbox Analysis is designed to answer these questions. The system:

1. Retrieves conversations within the specified time range
2. Uses an embedding model to cluster conversations by topic
3. Uses an LLM to analyze each conversation's intent, resolution status, and failure reasons
4. Generates an overall analysis summary with improvement suggestions

{% hint style="info" %}
**How this differs from similar features**

| Feature                                                                                 | When it runs                                         | Purpose                                                                                                |
| --------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| [Usage Analytics](https://docs.maiagent.ai/maiagent-user-guide/en/build/agent/usage.md) | Real-time statistics                                 | Quantitative KPIs at the Agent level (conversation character counts, counts, and satisfaction)         |
| Channel “Conversation Analysis” settings (a tab in each channel's settings)             | In real time, whenever a new message arrives         | Uses an LLM to automatically label conversations for routing and management                            |
| **Inbox Analysis, described on this page**                                              | **Manually triggered, retrospective batch analysis** | **Topic clustering, resolution rate analysis, and improvement suggestions for a specified date range** |
| {% endhint %}                                                                           |                                                      |                                                                                                        |

***

## How to Use It <a href="#how-to-use" id="how-to-use"></a>

{% stepper %}
{% step %}

### Open the Inbox Analysis Page <a href="#enter-page" id="enter-page"></a>

1. In the left menu, click “<mark style="color:blue;">Customer Service Conversations</mark>” → “<mark style="color:blue;">Channels</mark>”
2. Find the target channel in the channel list
3. Click the <mark style="color:blue;">📊 icon</mark> in the actions area on the right of that row (hover text: “Inbox Analysis”) to open the analysis page

The page lists all analysis reports previously created for this channel, including their date ranges, statuses, conversation counts, and processing times.

<figure><img src="https://1360999650-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6v6TNkkOQVfRYfcNirHL%2Fuploads%2Fgit-blob-ca2397bf2a39ccde7960854eb83724e9f230b6d3%2Finbox-analysis-01-list.png?alt=media" alt="Inbox Analysis page: analysis report list with columns for status, date range, total conversations, analyzed conversations, processing time, creation time, and actions"><figcaption><p>Inbox Analysis page: reports for this channel, with an option to add an analysis in the upper right</p></figcaption></figure>
{% endstep %}

{% step %}

### Create an Analysis <a href="#create-analysis" id="create-analysis"></a>

Click “<mark style="color:blue;">New Analysis</mark>” in the upper right and complete these required fields:

* **Date Range**: Select the conversation period to analyze (future dates are not allowed)
* **Analysis LLM**: The large language model used to analyze individual conversations
* **Embedding Model**: The embedding model used for topic clustering
* **Report Language**: The output language for the analysis summary (Traditional Chinese, Simplified Chinese, English, Japanese, or Korean)

<figure><img src="https://1360999650-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6v6TNkkOQVfRYfcNirHL%2Fuploads%2Fgit-blob-9381ae705c2fa3e43c4ed4f3d836aa92f1cfbebb%2Finbox-analysis-02-create-modal.png?alt=media" alt="New Analysis dialog: four required fields for date range, analysis LLM, embedding model, and report language, with advanced settings collapsed"><figcaption><p>New Analysis: complete the four required fields</p></figcaption></figure>

Expand “<mark style="color:blue;">Advanced Settings</mark>” to adjust the sampling strategy (see “Advanced Settings Explained” below).

<figure><img src="https://1360999650-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6v6TNkkOQVfRYfcNirHL%2Fuploads%2Fgit-blob-e1cefa856afa611e3c242e20d6a6e81393f2dfdc%2Finbox-analysis-03-create-modal-advanced.png?alt=media" alt="New Analysis dialog with advanced settings expanded: samples per cluster, maximum samples, concurrency, and maximum conversations to retrieve"><figcaption><p>Advanced settings: adjust sampling and concurrency parameters</p></figcaption></figure>

After you click “<mark style="color:blue;">Confirm</mark>,” the system runs the analysis asynchronously in the background. Its status progresses from “Pending” → “Processing” → “Completed.”
{% endstep %}

{% step %}

### View the Report <a href="#view-report" id="view-report"></a>

Once the status changes to “Completed,” click the report's “<mark style="color:blue;">View Report</mark>” icon (👁) to open the report details page.

The report displays content based on the available conversation data, including a data overview, resolution status, topic clusters, top questions, performance metrics, an analysis summary, and per-conversation analysis. Download PDF or Excel files from the upper right of the report. See [Download a PDF Report](#download-pdf) and [Download an Excel Report](#download-excel).

<figure><img src="https://1360999650-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6v6TNkkOQVfRYfcNirHL%2Fuploads%2Fgit-blob-ff67b37a8ce5e219b90591df99ff79799e8a801c%2Finbox-analysis-04-report-detail.png?alt=media" alt="Analysis report details page"><figcaption><p>Lower section of the analysis report details page: top questions, performance by topic, and analysis settings</p></figcaption></figure>
{% endstep %}
{% endstepper %}

***

## Report Contents <a href="#report-content" id="report-content"></a>

### 1. Data Overview <a href="#data-overview" id="data-overview"></a>

| Metric              | Description                                                                       |
| ------------------- | --------------------------------------------------------------------------------- |
| Total Conversations | Total conversations on this channel within the date range                         |
| Analyzed            | Number of conversations actually analyzed by the LLM (limited by Maximum Samples) |
| Sampling Rate       | Analyzed / Total Conversations (percentage)                                       |
| Processing Time     | Seconds from the start of this analysis task to completion                        |

{% hint style="info" %}
When the total conversation count exceeds Maximum Samples, the system selects representative conversations after clustering to keep costs and processing time under control.
{% endhint %}

### 2. Resolution Status <a href="#resolution-status" id="resolution-status"></a>

The key metric, **Resolution Rate**, is calculated as follows:

```
Resolution Rate = (Resolved Count + Partially Resolved Count) ÷ Analyzed Count
```

* **Resolved** (green): The Agent fully answered the user's question
* **Partially Resolved** (orange): The answer covers part of the question or is on the right track but incomplete
* **Unresolved** (red): The Agent could not answer or answered incorrectly

Resolution rates above 50% appear in green, while rates below 50% appear in red, providing an immediate health indicator.

### 3. Performance Metrics <a href="#performance-metrics" id="performance-metrics"></a>

Shows the Agent's computational performance during this period:

* **Total Token Usage**: Input + Output tokens combined
* **Average Tokens per Turn**: Average tokens consumed per conversation turn
* **Average Response Time**: Total time from receiving a message to completing the response
* **Average Time to First Token (TTFT)**: Time from receiving a message to starting to output the first token

### 4. Topic Cluster Distribution <a href="#topic-clusters" id="topic-clusters"></a>

Conversations are clustered by semantic similarity. A horizontal bar chart shows each topic's frequency and percentage. Outlier conversations are classified as “Noise / Outliers.”

This view helps you quickly identify the topics users ask about most often.

### 5. Resolution / Failure Reason Distribution <a href="#failure-reasons" id="failure-reasons"></a>

The system labels each analyzed conversation with a status or failure reason, summarized below:

| Reason                          | Description                                                     |
| ------------------------------- | --------------------------------------------------------------- |
| Resolved                        | The Agent successfully answered                                 |
| Partially Resolved              | The answer is on the right track but incomplete                 |
| Knowledge Base Gap              | Relevant content is missing from the knowledge base             |
| Overly Restrictive Instructions | Role instructions blocked a reasonable response                 |
| Tool Failure                    | A tool failed to execute or was not triggered                   |
| Unclear User Intent             | The user's message is vague and its intent cannot be determined |
| Off Topic                       | The user's question is outside the Agent's scope                |
| Hallucinated Response           | The AI generated nonexistent or incorrect information           |
| Unknown                         | Cannot be classified into any of the categories above           |

Use this distribution to guide improvements: many knowledge base gaps → add content; many overly restrictive instructions → adjust role instructions; many hallucinations → strengthen the knowledge base and instruction constraints. This is the only automated way to identify knowledge gaps: the platform does not separately record or export questions for which retrieval returned no results. Use this analysis's sampling results together with [Per-Conversation Analysis](#per-conversation-analysis) to fill gaps in the knowledge base.

### 6. Top Questions <a href="#top-questions" id="top-questions"></a>

A list of topics sorted by frequency, showing each topic's **occurrence count** and **resolution rate**. Expand a row to see example intents for that topic and quickly confirm whether the clustering matches your expectations.

### 7. Performance by Topic <a href="#performance-by-topic" id="performance-by-topic"></a>

Breaks down tokens and processing time by topic to identify expensive or slow topics:

* Average Input / Output / Total tokens
* Average response time
* Average time to first token

If token usage is unusually high for a topic, consider optimizing role instructions or knowledge base retrieval. If a topic takes a long time, it may involve many tool calls or lengthy content.

### 8. Analysis Summary <a href="#synthesis" id="synthesis"></a>

A Markdown report generated by the LLM from all analysis results, typically including:

* Overall performance summary
* Main problem patterns
* Priority improvement suggestions
* Specific areas to adjust

It appears in a collapsed section by default. Click the heading to expand and read it.

### 9. Per-Conversation Analysis <a href="#per-conversation-analysis" id="per-conversation-analysis"></a>

Expand this section to review each analyzed conversation, including:

* **User Intent**: What the user actually wants to ask
* **Topic**: The topic cluster the conversation belongs to
* **Resolution Detail**: A detailed description of the resolution status
* **Suggestions**: Specific ways to improve this conversation

***

## Download a PDF Report <a href="#download-pdf" id="download-pdf"></a>

When a report's status is “Completed,” a <mark style="color:blue;">Download PDF</mark> button appears alongside <mark style="color:blue;">Download Excel</mark> in the upper right of the report details page. The PDF layout follows the sections on the report details page, making it suitable for forwarding to managers or adding to presentations without having to compile screenshots yourself.

<figure><img src="https://1360999650-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6v6TNkkOQVfRYfcNirHL%2Fuploads%2Fgit-blob-bca388d573a05fa24db6a9b0073e41a389e316e6%2Finbox-analysis-08-download-pdf.png?alt=media" alt="Download PDF and Download Excel buttons in the upper right of the report details page, with the data overview, resolution status, and performance metrics below"><figcaption><p>Upper right of the report details page: Download PDF and Download Excel</p></figcaption></figure>

The PDF includes, in order:

* Report header: channel name, analysis period, report language, and report generation time
* Data overview, resolution status, and performance metrics
* Topic clusters, resolution and failure reasons, top questions, and performance by topic
* Summary and improvement suggestions

{% hint style="info" %}

* The PDF **does not include per-conversation analysis**. Its last line reminds you to “Download Excel for per-conversation details.” Download Excel to inspect individual conversations.
* Section headings use the report language. The filename follows the same rules as Excel (channel name + report title + analysis period), with a .pdf extension.
* Only one format can be downloaded at a time: while the PDF is being generated, <mark style="color:blue;">Download Excel</mark> is temporarily disabled, and vice versa. Both buttons become available again after the download finishes.
* Neither download button appears while the report is still being generated or if it has failed. If an older report lacks data for a section, that section in the PDF displays “No Data.”
* Any member who can view the report can download it without additional permissions.
  {% endhint %}

### Scenario: Share Analysis Results with a Manager Before a Monthly Meeting <a href="#pdf-use-case" id="pdf-use-case"></a>

At the start of each month, customer service manager Lin reports on Web Chat service at the operations meeting. She creates an analysis report for the previous month, waits for its status to become “Completed,” opens the report details, and clicks <mark style="color:blue;">Download PDF</mark>. She gets a report file laid out just like the screen and attaches it directly to the meeting invitation email. She also clicks <mark style="color:blue;">Download Excel</mark> to give the customer service team lead the per-conversation analysis for following up on individual cases.

***

## Download an Excel Report <a href="#download-excel" id="download-excel"></a>

When a report's status is “Completed,” the <mark style="color:blue;">Download Excel</mark> button appears in the upper right of the report details page. Click it to download an .xlsx file. The filename includes the channel name and analysis period, making it easy to organize the data or perform further analysis in your own way.

| Sheet                          | Contents                                                                                                                                         |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Summary                        | Analysis settings (channel, analysis period, report language, report generation time), data overview, resolution status, and performance metrics |
| Topic Clusters                 | Topic, conversation count, percentage                                                                                                            |
| Resolution and Failure Reasons | Reason, count, percentage                                                                                                                        |
| Top Questions                  | Rank, topic, frequency, resolution rate, example intents                                                                                         |
| Performance by Topic           | Topic, frequency, average input / output / total tokens, average response time, average time to first token                                      |
| Per-Conversation Analysis      | Conversation ID, conversation time, user intent, topic, resolution status, reason, details, suggestions                                          |
| Analysis Summary               | Report summary and improvement suggestions, with each paragraph in a separate row                                                                |

{% hint style="info" %}

* Percentages and times are stored as numeric values (time in seconds), so you can sort and calculate directly after downloading.
* Sheet names and column names follow the report language.
* The download button does not appear while the report is still being generated or if it has failed. If an older report lacks data for a section, that sheet displays “No Data.” If there is no analysis summary, the Analysis Summary sheet also displays “No Data.”
* Any member who can view the report can download it without additional permissions.
  {% endhint %}

***

## Advanced Settings Explained <a href="#advanced-config" id="advanced-config"></a>

Advanced settings control sampling and execution strategies. Start with the defaults, then adjust as needed:

| Parameter                          | Default | Description                                                                                                              |
| ---------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------ |
| Samples per Cluster                | 10      | Maximum conversations from each topic cluster to send to the LLM for analysis                                            |
| Maximum Samples                    | 150     | Maximum conversations analyzed in the entire report (caps costs)                                                         |
| Concurrency                        | 8       | Number of concurrent LLM analyses; higher values are faster but consume more resources at once                           |
| Maximum Conversations to Retrieve  | —       | Upper limit on the total conversations retrieved from the database                                                       |
| Minimum Conversation Message Count | —       | Conversations with fewer messages than this value are filtered out to avoid analyzing short, uninformative conversations |

{% hint style="warning" %}
**Cost reminder**: Analysis consumes LLM token quota (Input + Output + Embedding). The system checks your organization's quota before analysis; tasks cannot be created if the quota is insufficient.
{% endhint %}

***

## Use Cases <a href="#use-cases" id="use-cases"></a>

### Scenario 1: Quarterly Quality Review <a href="#case-quarterly-review" id="case-quarterly-review"></a>

**Need**: A manager wants to understand the conversation quality and main issues on the Web Chat channel during the previous quarter.

**Steps**:

1. Select the previous quarter as the date range
2. Set Maximum Samples to 300 to increase the sample size
3. Run the analysis and focus on Resolution / Failure Reason Distribution and Top Questions
4. Schedule improvements for frequent causes, such as knowledge base gaps and overly restrictive instructions

### Scenario 2: Assess the Impact of a Version Update <a href="#case-version-impact" id="case-version-impact"></a>

**Need**: You adjusted role instructions last week and want to know whether conversation quality improved.

**Steps**:

1. Create two analysis reports, one for the week before the adjustment and one for the week after
2. Compare their resolution rates and failure reason distributions
3. Check whether specific failure types decreased

### Scenario 3: Fill Knowledge Base Gaps Through Topic Clustering <a href="#case-knowledge-gap" id="case-knowledge-gap"></a>

**Need**: You have identified frequently asked questions and want to systematically expand the knowledge base.

**Steps**:

1. Analyze conversations from the past month
2. From Top Questions, select topics with high frequency and low resolution rates
3. Expand Example Intents to confirm how users actually phrase their questions
4. Add the corresponding FAQs or documents to the knowledge base

***

## Frequently Asked Questions <a href="#faq" id="faq"></a>

### Q: Why Were Some Conversations Not Analyzed? <a href="#faq-why-not-all-analyzed" id="faq-why-not-all-analyzed"></a>

The system filters conversations based on Maximum Samples and Minimum Conversation Message Count, then selects representative samples after clustering. If the total conversation volume is large, only representative samples are analyzed to control costs.

### Q: How Is the Resolution Rate Determined? <a href="#faq-resolution-rate" id="faq-resolution-rate"></a>

The analysis LLM **directly classifies each analyzed conversation into one of nine statuses**: Resolved, Partially Resolved, Knowledge Base Gap, Overly Restrictive Instructions, Tool Failure, Unclear User Intent, Off Topic, Hallucinated Response, or Unknown.

Resolution rate formula:

```
Resolution Rate = (Resolved + Partially Resolved) ÷ Total Analyzed
```

Both Resolved and Partially Resolved count toward the numerator. The remaining seven statuses are treated as unresolved.

### Q: Can I Create Multiple Analyses for the Same Channel at Once? <a href="#faq-concurrent-analyses" id="faq-concurrent-analyses"></a>

Yes, but **date ranges must not overlap with an analysis in progress**. If you see “This channel already has an analysis in progress with an overlapping date range,” wait for that analysis to finish or select a different date range.

### Q: How Long Does an Analysis Take? <a href="#faq-duration" id="faq-duration"></a>

This depends on the conversation volume, maximum samples, concurrency, and selected models. In general, 150 samples take a few minutes to over ten minutes. The default processing time limit is 110 minutes; tasks that exceed it are automatically marked as failed.

### Q: What Should I Do if an Analysis Fails? <a href="#faq-failure" id="faq-failure"></a>

If the status is “Failed,” check the Error Message column in the report list. Common messages and remedies:

* “Analysis timed out — the worker may have crashed during processing.” Processing exceeded the time limit: lower Maximum Samples to reduce the analysis volume, then retry
* “Analysis task was never picked up by a worker.” The task was not assigned: try again later
* Other messages: take a screenshot and contact MaiAgent support for troubleshooting

{% hint style="info" %}
If your **quota is insufficient**, analysis creation is blocked immediately with a message. It will not appear as a report with a “Failed” status.
{% endhint %}

### Q: Why Can't My Account See Inbox Analysis? <a href="#faq-permission" id="faq-permission"></a>

Inbox Analysis is a sub-permission of Customer Service Conversations access. By default, any MaiAgent role with Customer Service Conversations access includes this feature.

If you cannot see it, possible reasons include:

* Your MaiAgent role does not have Customer Service Conversations access
* Your organization administrator has disabled the Inbox Analysis sub-permission for your role

Without this permission, the <mark style="color:blue;">Channels</mark> list does not show the analysis icon. Opening the analysis page directly through a link displays a 403 error: “Sorry, you do not have permission to view this page.” Contact your organization administrator to check and adjust permissions on the [Role Permissions Management](https://docs.maiagent.ai/maiagent-user-guide/en/org/roles/role-permission.md#request-permission) page.

***

## Notes <a href="#notes" id="notes"></a>

{% hint style="info" %}
**Permissions**

* Inbox Analysis is a sub-permission of Customer Service Conversations access, included by default in all MaiAgent roles with Customer Service Conversations permission
* Organization administrators can adjust feature access for different roles on the [Role Permissions Management](https://docs.maiagent.ai/maiagent-user-guide/en/org/roles/role-permission.md) page
* Members with this permission can create and view all analysis reports within their organization
  {% endhint %}

{% hint style="warning" %}
**Cost reminders**

* Each analysis consumes LLM Input / Output tokens and Embedding tokens
* Check the date range and maximum samples in the modal before creating an analysis
* Adjust Maximum Samples in advanced settings to cap costs
  {% endhint %}

{% hint style="danger" %}
**Data protection**

* Conversation content is sent to the selected LLM for processing during analysis
* Choose a compliant model according to your organization's data processing policies
  {% endhint %}

***

## Related Features <a href="#related" id="related"></a>

{% hint style="info" %}
**Further reading**

* [Usage Analytics](https://docs.maiagent.ai/maiagent-user-guide/en/build/agent/usage.md): Quantitative KPI dashboard at the Agent level
* [Channel Search](https://docs.maiagent.ai/maiagent-user-guide/en/serve/daily-operations/reply/inbox-search.md): Quickly find specific conversations
* [Conversation Viewer](https://docs.maiagent.ai/maiagent-user-guide/en/serve/daily-operations/reply/conversation-viewer.md): View the full context of an individual conversation
* [Evaluation Insights Report](https://docs.maiagent.ai/maiagent-user-guide/en/build/agent-ops/evaluation-insights.md): Automated insights after AgentOps batch testing
  {% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.maiagent.ai/maiagent-user-guide/en/serve/daily-operations/reply/inbox-analysis.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
