Search Testing

Before officially launching LLM, you can preview its functionality through search testing to verify the relevance and content of the data retrieved by LLM

Why is Search Testing Needed?

1. Ensure Response Accuracy

• AI assistant's response quality directly impacts customer satisfaction

• Incorrect or irrelevant answers damage brand trust

1. Prevent Information Gaps

• Avoid overlooking important product details or safety instructions

• Optimize knowledge base utilization efficiency

1. Improve Response Consistency

• Ensure consistent answers to the same questions at different times

• Avoid response instability due to knowledge base updates

Through search testing and snippet preview features, you can instantly review the quality of data segments used by the AI assistant to ensure your Q&A standards.

How to Begin?

1. Enter the knowledge base page, select the knowledge base you want to view and enter settings

1. Click on search testing to enter the page

Enter the content you want to search in the dialog box: "What should I pay attention to when choosing a campsite?", after pressing the search button, the AI assistant will search for the most relevant documents based on your conversation content.

1. Review Search Results

The right side will show the segments that the AI assistant will retrieve when answering, as shown in the image there are 12 segments (for retrieval segment quantity settings, please refer to: How to Create a Knowledge Base: Basic Settings)

The right side shows the retrieved content used for answering this question, each segment displays:

• Word count: Number of words covered in the segment

• Hit count: Number of times the AI assistant referenced this segment when answering (only counts formal Q&A, test Q&A is not included in hit count calculations)

Higher hit counts indicate that users ask more questions about this segment.

Search testing records past test Q&As, you can quickly click on the history to retest and ensure that the same topics maintain or improve data reference quality.

Check Knowledge Segment Quality

Through search testing, you can determine which segments the AI assistant uses for reasoning and then check if these segments need optimization, such as:

If you find many blank segments in the search results, and these blank segments all belong to the "Beginner Camping.pdf" file, you can then check if the contents of "Beginner Camping.pdf" are all correct and update them in real-time before official launch.

How Search Testing Helps You

Regular Testing of Important Questions

• Recommend frequent testing of core business-related questions

• Conduct retesting after knowledge base updates

• Record and compare test result improvements

Identify Content Gaps

• Discover knowledge gaps: Test common questions to find queries with no matching results

• Example: Testing "tent waterproof rating explanation" reveals no related segments → Need to add waterproof specification documents

Evaluate Content Completeness

• Check answer depth: Verify if AI answers cover all aspects of the question

• Example: Query "camping safety precautions" → Check if it covers weather, wildlife, fire safety, and other aspects

Question Testing Techniques

You can:

• Ask the same question in different ways

• Test complex scenarios and multi-condition queries

• Verify retrieval effectiveness for technical terms and product names

Through these testing techniques, comprehensively evaluate the accuracy and completeness of responses.