IVR Customer Service Intent Recognition

IVR (Interactive Voice Response) Customer Service Intent Recognition refers to a technology application that combines AI voice processing technology, allowing users to interact with customer service systems through voice commands and automatically recognizing customer intent to provide corresponding services. In Taiwan, such systems are widely used in banking, telecommunications, healthcare, and other industries to improve service efficiency and customer experience.

Core Features and Process

1. Receive Voice Input After the customer calls, the system plays an automated voice menu (e.g., "Please briefly describe your needs, and we will provide assistance."). The customer does not need to press buttons to select options but instead directly expresses their needs through voice.

2. Speech Recognition (ASR) The system uses Automatic Speech Recognition technology to convert speech into text. For example, if the customer says "I want to check my bill", the system converts the content into text input.

3. Natural Language Understanding (NLU) The system uses Natural Language Understanding technology to analyze semantics and determine the user's actual needs. For example:

   • Vocabulary analysis: Keywords like "check" and "bill" indicate the need is related to billing.

   • Intent recognition: Determines the customer's purpose is to "check bill".

4. Response and Routing The system provides corresponding services based on semantic analysis results, with the following possible options:

   • Direct Response: If the need can be handled automatically, e.g., "Your bill amount is NT$1,200, and the payment deadline is December 15th."

   • Transfer to Agent: If the need is more complex, the system automatically transfers to the corresponding department's customer service representative while providing a semantic summary to reduce redundant communication.

Technical Challenges and Limitations

1. Difficulty Understanding Diverse Expressions

   • Users' language expressions may be non-standard, e.g., "What's going on with my bill?" or "What about the payment issue?" — such semantics can be somewhat ambiguous, and models like BERT sometimes cannot precisely determine specific needs.

   • In Taiwan, there is also the challenge of multi-language and multi-dialect mixing (Mandarin, Taiwanese, Hakka), and language model support for these languages is still insufficient.

2. Limitations in Intent Granularity

   • While modern NLP models can handle large volumes of text data, they cannot fully master specialized knowledge or unique intents in certain industries. For example, "I want to know the specific date of my last payment" may require connecting to different systems to answer correctly.

   • Even though BERT handles shorter conversation fragments well, long sentences or complex semantic expressions can confuse the model.

3. Data Bias and Incomplete Corpora

   • Training language models requires massive localized corpora. If data is insufficient or biased toward a single expression pattern, the model's adaptability to special contexts will be inadequate. For example, Taiwan-specific language habits like "top-up" or "skip number" may lack sufficient contextual corpora in the model.

4. Context and Memory Limitations

   • Customer conversations typically have contextual relevance, e.g., "About the payment mentioned earlier, I have another question" — such multi-turn conversations require the system to remember previous intents. Current NLP models have limited performance in this area.

   • If intent determination is wrong, users may have to restate their needs, causing frustration.

5. Low Error Tolerance

   • Customers have limited patience with customer service systems. If the voice system makes incorrect determinations, customers may become frustrated and ultimately request to speak with a human agent.

Industry Status

Currently, many enterprises' voice customer service systems still use traditional button-based selection processes. Taking SinoPac Bank's voice customer service system as an example, its design thoroughly considers business diversity, providing multi-level menu options to guide users. However, there is still significant room for optimization and improvement in user interaction experience.

The emergence of LLM (Large Language Models) and RAG (Retrieval-Augmented Generation) has brought revolutionary changes to semantic recognition and the overall IVR system, making voice customer service systems smarter, more precise, and more adaptive, overcoming many limitations of traditional NLP technologies.

永豐銀行 Bank SinoPac - 客服中心 – 連絡客服 – 語音服務bank.sinopac.com

Solution

The following will guide you step by step through building a powerful semantic recognition assistant using MaiAgent's robust and precise LLM and RAG capabilities.

Steps

1. Data Preparation

In the past, BERT required large amounts of labeled data for NLP tasks (such as intent recognition, sentiment analysis, etc.). Labeled data was typically created manually — for example, annotating sentences with intent categories or keywords — which was both time-consuming and expensive. Even with high-quality labeled data, the model's generalization ability was insufficient. When business needs or usage habits changed, the model needed to be re-labeled and retrained, which was a lengthy process.

The advantage of LLM and RAG is that they can fully combine generative language capabilities with the dynamic nature of real-time retrieval, eliminating dependence on labeled data, improving semantic recognition accuracy, reducing development and maintenance costs, and greatly improving user experience. This technology combination sets new industry standards for intelligent customer service and voice interaction, and is a key driver for future automation and personalized services.

The introduction of LLM and RAG has greatly simplified the labeled data preparation process. Now, you simply need to organize data into an Excel spreadsheet, list the intent categories, and upload this banking customer service list to the MaiAgent AI assistant's knowledge base to support the intelligent semantic recognition system.

Banking service list (for intent recognition)

2. Define Role Instructions

# Role
You are MaiAgent Bank's semantic understanding robot.

# Output Format
Based on the user's conversation, understand the customer's desired service intent, determine the user's desired service intent from the knowledge base, and output it.

When the intent is clear, output only one intent; if there are multiple similar intents that cannot be distinguished, list up to 3 of the closest intents.

<example>
-<Code>:<Category> - <Subcategory>
</example>

<example>
-<Code>:<Category> - <Subcategory>
-<Code>:<Category> - <Subcategory>
...
</example>

<example>
N/A
</example>

# Output Restrictions
- Reply in Traditional Chinese
- Do not answer questions outside the knowledge base scope
- Output only the text within <example> and </example>, do not include other descriptions
- Output should not include <example> and </example>
- Answer based on knowledge base data; if unable to determine intent, use the text within the <example> below

3. Start Using

Usage Examples

Single-Turn Conversation

Intent recognition example

Multi-Turn Conversation

LLM and RAG technology resolves the bottleneck of multi-turn conversation intent recognition that was difficult with BERT.

Intent recognition multi-turn conversation example