> For the complete documentation index, see [llms.txt](https://docs.maiagent.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/conversations/inbox-analysis.md). # Conversation Platform Analytics Inbox Analysis can analyze real conversations within a specific platform and date range, using AI to automatically categorize topics, determine resolution status, and compile actionable improvement suggestions to help you continuously optimize your AI assistant's conversation quality. ## What is Inbox Analysis? Once customer service conversations accumulate to a certain volume, looking at totals or satisfaction scores alone isn't enough โ€” you want to know: * **What are users actually asking?** What are the most frequently asked topics? * **How many conversations were truly resolved?** Which types of issues have low resolution rates? * **Why weren't they resolved?** Is it missing knowledge base content, overly restrictive instructions, or tool failures? * **How are response times and costs distributed?** Which topics take longer and consume more tokens? Inbox Analysis is designed to answer these questions. The system will: 1. Retrieve conversations within the specified time range 2. Use embedding models for topic clustering 3. Use LLM to analyze each conversation's intent, resolution status, and failure reasons 4. Generate a comprehensive analysis summary (including improvement suggestions) {% hint style="info" %} **Differences from Similar Features** {% endhint %} | Feature | Trigger Timing | Purpose | | --------------------------------------------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------ | | [Usage Analytics](/maiagent-user-guide/maiagent-user-guide-en/org/usage.md) | Real-time statistics | AI assistant-level quantitative KPIs (conversation word count, frequency, satisfaction) | | Inbox "Conversation Analysis" settings (tab within each inbox's settings) | Real-time, on each new incoming message | Use LLM to automatically tag conversations for routing and management | | **Inbox Analysis (this page)** | **Manual trigger, batch review** | **Topic clustering, resolution rate analysis, and improvement suggestions for a specified date range** | *** ## How to Use {% stepper %} {% step %} ### Navigate to the Inbox Analysis Page 1. Click "Customer Service" โ†’ "Inboxes" in the left menu 2. Find the target inbox in the inbox list 3. Click the ๐Ÿ“Š icon on the right side of that row (hover shows "Inbox Analysis") to enter the analysis page The page displays all analysis reports previously created for this inbox, including date range, status, conversation count, processing time, and other information. {% endstep %} {% step %} ### Create a New Analysis Click the "New Analysis" button in the upper right corner and fill in the following required fields: * **Date Range**: Select the conversation period to analyze (future dates cannot be selected) * **Analysis LLM**: The large language model used to analyze each conversation * **Embedding Model**: The embedding model used for topic clustering * **Report Language**: The output language for the analysis summary (Traditional Chinese, Simplified Chinese, English, Japanese, Korean) Expand "Advanced Settings" to adjust the sampling strategy (see "Advanced Settings" section below). After clicking "Confirm", the system will run the analysis asynchronously in the background. The status will progress from "Pending" โ†’ "Processing" โ†’ "Completed". {% endstep %} {% step %} ### View the Report Once the status changes to "Completed", click the "View Report" icon (๐Ÿ‘) on that report to enter the report detail page. The report content varies depending on conversation data richness, and includes data overview, resolution status, topic clusters, top questions, performance metrics, analysis summary, and per-conversation analysis. {% endstep %} {% endstepper %} *** ## Report Content ### 1. Data Overview | Metric | Description | | ------------------- | ------------------------------------------------------------------------------- | | Total Conversations | Total number of conversations on the inbox within the date range | | Analyzed | Number of conversations actually analyzed by LLM (limited by "Max Sample Size") | | Sample Rate | Analyzed / Total Conversations (percentage) | | Processing Time | Time in seconds from start to completion of this analysis task | {% hint style="info" %} When the total number of conversations exceeds the "Max Sample Size", the system uses a post-clustering sampling strategy to select representative conversations for analysis, controlling cost and time. {% endhint %} ### 2. Resolution Status The core metric "**Resolution Rate**" formula: ``` Resolution Rate = (Resolved + Partially Resolved) รท Analyzed ``` * **Resolved** (green): AI assistant fully answered the user's question * **Partially Resolved** (orange): Answer was partially correct or in the right direction but incomplete * **Unresolved** (red): Unable to answer or answered incorrectly Resolution rates above 50% are displayed in green, below 50% in red, serving as an instant health indicator. ### 3. Performance Metrics Shows the AI assistant's computational performance during this period: * **Total Token Usage**: Input + Output tokens combined * **Average Tokens per Turn**: Average token consumption per conversation turn * **Average Response Time**: Total time from receiving a message to completing the response * **Average Time to First Token (TTFT)**: Time from receiving a message to outputting the first token ### 4. Topic Cluster Distribution All conversations are clustered by "semantic similarity" and displayed as horizontal bar charts showing the frequency and proportion of each topic. Outlier conversations are categorized as "Noise / Outliers". This view helps quickly identify the most frequently asked topics. ### 5. Resolution / Failure Reason Distribution The system labels each analyzed conversation with a status or failure reason, summarized as follows: | Reason | Description | | ------------------------------- | -------------------------------------------------------- | | Resolved | AI assistant successfully answered | | Partially Resolved | Answer was in the right direction but incomplete | | Knowledge Gap | Knowledge base lacks relevant content | | Overly Restrictive Instructions | Role instructions blocked a reasonable response | | Tool Failure | Tool execution error or not triggered | | Unclear User Input | User message was vague or intent could not be determined | | Off-Topic | User's question was outside the assistant's scope | | Hallucination | AI generated non-existent or incorrect information | | Unknown | Cannot be categorized into the above | This distribution directly guides subsequent improvement actions: many knowledge gaps โ†’ add content; many overly restrictive instructions โ†’ adjust role instructions; many hallucinations โ†’ strengthen knowledge base and instruction constraints. ### 6. Top Questions Ranking A topic list sorted by "frequency", showing each topic's **occurrence count** and **resolution rate**. Expanding any row reveals example intents for that topic, helping you quickly verify whether the clustering matches expectations. ### 7. Performance Distribution by Topic Breaks down tokens and response times by topic to identify "expensive" or "slow" topics: * Average Input / Output / Total Tokens * Average Response Time * Average Time to First Token If a topic has abnormally high token usage, consider optimizing role instructions or knowledge base retrieval; if a topic takes long, it may be due to multiple tool calls or lengthy content. ### 8. Analysis Summary A Markdown report generated by LLM synthesizing all analysis results, typically including: * Overall performance summary * Major issue patterns * Priority improvement suggestions * Specific adjustment directions Displayed as a collapsed section by default; click the title to expand and read. ### 9. Per-Conversation Analysis When expanded, you can review each analyzed conversation individually, with each entry containing: * **User Intent**: What the user actually wanted to ask * **Topic**: The assigned topic cluster * **Resolution Detail**: Detailed description of the resolution status * **Suggestions**: Specific optimization directions for that conversation *** ## Advanced Settings Advanced settings control sampling and execution strategies. It's recommended to start with default values and adjust as needed: | Parameter | Default | Description | | --------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------- | | Samples per Cluster | 10 | Maximum conversations per topic cluster sent to LLM for analysis | | Max Sample Size | 150 | Maximum conversations analyzed in the entire report (cost ceiling) | | Concurrency | 8 | Number of parallel LLM analysis processes; higher = faster but more resource-intensive | | Max Retrieved Conversations | โ€” | Upper limit of conversations fetched from the database | | Min Conversation Messages | โ€” | Conversations with fewer messages than this value are filtered out (avoids analyzing meaningless short conversations) | {% hint style="warning" %} **Cost Reminder**: Analysis consumes LLM token quota (Input + Output + Embedding). The system checks organization quota before analysis; tasks cannot be created if quota is insufficient. {% endhint %} *** ## Use Cases ### Scenario 1: Quarterly Quality Review **Need**: A manager wants to understand last quarter's conversation quality and main issues on the Web Chat inbox. **Steps**: 1. Set the date range to the previous quarter 2. Set Max Sample Size to 300 (larger sample) 3. Run the analysis, focusing on "Resolution / Failure Reason Distribution" and "Top Questions Ranking" 4. Schedule improvement tasks based on high-frequency reasons such as knowledge gaps and overly restrictive instructions ### Scenario 2: Impact Assessment After Version Update **Need**: Role instructions were adjusted last week; want to know if conversation quality improved. **Steps**: 1. Create two analysis reports for "one week before the adjustment" and "one week after the adjustment" 2. Compare resolution rates and failure reason distributions between the two reports 3. Confirm whether specific failure types have decreased ### Scenario 3: Topic Clustering for Knowledge Base Completion **Need**: Discovered frequently asked questions and want to systematically supplement the knowledge base. **Steps**: 1. Analyze conversations from the past month 2. From "Top Questions Ranking", pick high-frequency, low-resolution-rate topics 3. Expand "Example Intents" to confirm actual phrasings 4. Go back to the knowledge base and add corresponding FAQ or documents *** ## FAQ ### Q: Why weren't some conversations analyzed? The system filters based on "Max Sample Size" and "Min Conversation Messages", and performs representative sampling after clustering. If the total number of conversations is large, only representative samples are selected to control costs. ### Q: How is the resolution rate determined? The analysis LLM **directly classifies each analyzed conversation into one of 9 statuses**: Resolved, Partially Resolved, Knowledge Gap, Overly Restrictive Instructions, Tool Failure, Unclear User Input, Off-Topic, Hallucination, Unknown. Resolution Rate formula: ``` Resolution Rate = (Resolved + Partially Resolved) รท Total Analyzed ``` Both "Resolved" and "Partially Resolved" count toward the resolution rate numerator; the remaining 7 statuses are all considered unresolved. ### Q: Can multiple analyses be created for the same inbox simultaneously? Yes, but **date ranges cannot overlap with an in-progress analysis**. If you see the error "This inbox already has an in-progress analysis with overlapping date range", wait for that analysis to complete or select a different date range. ### Q: How long does the analysis take? It depends on conversation volume, max sample size, concurrency, and the selected model. Generally, 150 samples take approximately a few minutes to around ten minutes. The system's default processing time limit is 110 minutes; analyses exceeding this will be automatically marked as failed. ### Q: What if the analysis fails? If the status shows "Failed", check the "Error Message" column in the report list. Common messages and how to handle them: * "Analysis timed out โ€” the worker may have crashed during processing." Processing exceeded the time limit: reduce "Max Sample Size" to lower the analysis volume and retry * "Analysis task was never picked up by a worker." Task was not dispatched: try again later * For other messages: take a screenshot and report to MaiAgent support for investigation {% hint style="info" %} If there is **insufficient quota**, the analysis will be blocked at creation time with a prompt, and will not appear as a "Failed" report. {% endhint %} ### Q: Why can't I see "Inbox Analysis" on my account? "Inbox Analysis" is a sub-permission under the "Customer Service" access permission. By default, any MaiAgent role with "Customer Service" access permission will include this feature. If you cannot see it, it may be because: * Your MaiAgent role does not have "Customer Service" access permission * The organization administrator has disabled the "Inbox Analysis" sub-permission for your role Contact your organization administrator to check and adjust settings on the [Role Permission Management](/maiagent-user-guide/maiagent-user-guide-en/org/role-permission.md) page. *** ## Notes {% hint style="info" %} **Permission Notes** * "Inbox Analysis" is a sub-permission under the "Customer Service" access permission, included by default in all MaiAgent roles with "Customer Service" permission * Organization administrators can adjust this feature's availability for different roles on the [Role Permission Management](/maiagent-user-guide/maiagent-user-guide-en/org/role-permission.md) page * Members with this permission can create and view all analysis reports within their organization {% endhint %} {% hint style="warning" %} **Cost Reminder** * Each analysis consumes LLM Input / Output Tokens and Embedding Tokens * Before creating an analysis, confirm the date range and max sample size in the modal * Adjust the max sample size in advanced settings to control cost ceiling {% endhint %} {% hint style="danger" %} **Data Protection** * The analysis process sends conversation content to the selected LLM for processing * Choose a compliant model according to your organization's data processing policies {% endhint %} *** ## Related Features {% hint style="info" %} **Further Reading** * [Usage Analytics](/maiagent-user-guide/maiagent-user-guide-en/org/usage.md): Quantitative KPI dashboard at the AI assistant level * [Inbox Search](/maiagent-user-guide/maiagent-user-guide-en/conversations/inbox-search.md): Quickly search for specific conversations * [Conversation Viewer](/maiagent-user-guide/maiagent-user-guide-en/conversations/conversation-viewer.md): View the full context of a single conversation * [Evaluation Insights Report](https://github.com/Playma-Co-Ltd/maiagent-user-guide-gitbook/blob/main/en/channels/evaluation-insights.md): Automated insights after AgentOps batch testing {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.maiagent.ai/maiagent-user-guide/maiagent-user-guide-en/conversations/inbox-analysis.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.