Ctrlk
註冊/登入
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Custom Maximum Output Tokens

Customize the maximum response length for your AI assistant

Feature Introduction

The custom max output tokens feature allows you to precisely control the maximum length of each AI assistant response. By adjusting this setting, you can have the AI assistant generate brief summaries or detailed explanations based on different use cases, while effectively managing token consumption costs.

About Tokens

Tokens are the basic units used by AI models to process text. Generally speaking:

  • Chinese: 1 character is approximately 1.5-2 Tokens

  • English: 1 word is approximately 1-1.5 Tokens

  • Punctuation marks also count toward the token count

Configure Max Output Tokens

Navigate to Settings Page

  1. Click "AI Assistant" in the left menu

  2. Select the AI assistant you want to configure

  3. Click the "Settings" button

  4. Switch to the "Advanced Settings" tab

Adjust Token Limit

  1. Select Token Limit Mode

    • "Use Model Maximum": Directly apply the maximum output tokens supported by the selected model (default)

    • "Custom Amount": Specify the limit yourself

  2. Enter Custom Value (when "Custom Amount" is selected)

    • Enter the target value in the number input field

    • The available range depends on the selected model, and the screen will display the allowed range in real time (e.g., "Range: 512 - 64000 tokens" as shown below)

  3. Save Settings

    • Click the "Save" button

    • Settings take effect immediately without restarting the AI assistant

Advanced Settings: Max Output Tokens
AI assistant advanced settings page: Choose "Use Model Maximum" or "Custom Amount" to control the max output token limit

Important Notes

  • Setting the value too low may cause responses to be truncated, affecting completeness

  • Setting the value too high will increase token consumption costs

  • It is recommended to adjust based on actual needs and find the optimal balance through testing

Recommended Settings

Below are recommended token limit settings for different use cases:

Use Case
Recommended Tokens
Description

Quick Q&A

256 - 512

Suitable for quick answers, such as FAQ bots Approximately 100-200 Chinese characters

General Customer Service

512 - 1024

Balances detail and cost Approximately 200-400 Chinese characters

Professional Consultation

1024 - 2048

Provides detailed explanations and solutions Approximately 400-800 Chinese characters

Content Generation

2048 - 4096

Generates long-form articles or reports Approximately 800-1600 Chinese characters

Code Generation

1024 - 2048

Produces complete code snippets Including comments and explanations

Use Case Examples

Scenario 1: FAQ Quick Response Bot

Requirement: Common customer questions need short and clear answers, avoiding lengthy explanations

Configuration:

  • Max output tokens: 256-512

  • System prompt: "Please answer questions in a brief and clear manner, no more than 3 sentences"

  • Expected result: Quick responses, reduced token consumption

Example conversation:

Scenario 2: Technical Support Detailed Explanation

Requirement: Users encounter technical issues and need detailed step-by-step instructions

Configuration:

  • Max output tokens: 1024-2048

  • System prompt: "Please provide detailed step-by-step instructions to ensure users can follow along"

  • Expected result: Complete troubleshooting guide

Example conversation:

Scenario 3: Content Summary Generation

Requirement: Extract key summaries from long articles

Configuration:

  • Max output tokens: 512-768

  • System prompt: "Please organize key points into 3-5 bullet points, each no more than 2 sentences"

  • Expected result: Concise summaries that are easy to quickly understand

Relationship with Other Settings

System Prompt

Even with a high token limit, the system prompt can still constrain response length:

Model Selection

Different models support different maximum output token counts. When you switch the model used by the AI assistant, the customizable range will change accordingly, and the range hint displayed on the screen reflects the upper and lower limits of the currently selected model. When "Use Model Maximum" is selected, the system will automatically apply the maximum value for that model.

Cost Optimization Recommendations

Calculate Token Consumption

Token consumption per conversation includes:

  • Input Tokens: User question + conversation history + system prompt

  • Output Tokens: AI response (limited by this setting)

Example calculation:

Optimization Strategies

  1. Adjust by scenario

    • Use a low token limit for simple questions

    • Only increase the limit for complex questions

  2. Avoid unnecessary long responses

    • Explicitly require conciseness in the system prompt

    • Use multi-turn conversations instead of single long responses

  3. Regularly review usage

    • Use the "Usage Analytics" tab to view conversation word count and other usage trends

    • Adjust settings to optimize costs

For detailed cost calculations and optimization recommendations, please refer to Usage Calculation.

FAQ

Q: After setting the token limit, will the AI response be forcibly truncated?

A: Yes. When the AI-generated response reaches the token limit, it will stop at that point, which may result in incomplete sentences. Recommendations:

  • Set an appropriate limit (not too low)

  • Include in the system prompt: "End naturally before reaching the character limit"

Q: How do I check the average token usage of my AI assistant?

A: Go to the AI assistant "Settings" page and select the "Usage Analytics" tab. You can view trend charts by date range for "Conversation Word Count", "Conversation Count", "Message Count", "Average Conversation Interactions", and "User Satisfaction". The "Conversation Word Count" represents the cumulative word count generated by the assistant. Word count is positively correlated with token count and can serve as a reference for token consumption.

Usage Analytics tab
AI assistant "Usage Analytics" tab: View trend charts by date range for conversation word count, conversation count, message count, average conversation interactions, and user satisfaction

Q: Does setting a higher token limit slow down response speed?

A: There is a slight impact. Longer responses require more generation time, but the difference is usually minimal (within a few seconds). The main factors affecting speed are:

  • Model selection

  • Network connection speed

  • Server load

Q: Do different languages require different token limit settings?

A: It is recommended to adjust accordingly:

  • Chinese: The same amount of text consumes more Tokens, so consider slightly increasing the limit

  • English: Token efficiency is higher, so a lower limit can be used

  • Multi-language environments: It is recommended to set a higher limit to ensure flexibility

Q: Does the token limit affect knowledge base retrieval results?

A: It does not directly affect retrieval, but it affects how retrieval results are presented:

  • High token limit: Can reference more knowledge base segments

  • Low token limit: Only references the most relevant segments

Testing and Adjustment

Testing Process

  1. Set Initial Value

    • Start with the recommended value (e.g., 1024)

  2. Conduct Test Conversations

    • Enter typical user questions

    • Observe response length and completeness

  3. Evaluate Results

    • Is the response complete?

    • Is there unnecessary verbosity?

    • Is token consumption reasonable?

  4. Adjust Gradually

    • Too long → Lower the token limit

    • Too short/truncated → Increase the token limit

  5. Continuous Monitoring

    • Regularly review usage

    • Fine-tune based on actual needs

Testing Examples

Related Features

Extended Features

PreviousAI Customer Service Quality ManagementNextKnowledge Base Overview

Last updated

Was this helpful?