ICAP Content Validation Integration
This document explains how the MaiAgent platform integrates the ICAP (Internet Content Adaptation Protocol) to implement security scanning and validation of user-uploaded files and conversation content, protecting against malware and inappropriate content.
1. What is ICAP?
ICAP (Internet Content Adaptation Protocol) is a lightweight protocol for forwarding content to external services for processing, such as virus scanning, content filtering, and data loss prevention.
1.1 Core Value of ICAP
Security
Users may upload malware or inappropriate content
Automatically scans and blocks malicious files and content
Compliance
Difficult to meet enterprise security policy requirements
Integrates with enterprise-grade antivirus and DLP solutions
Flexibility
Must develop content detection logic in-house
Uses a standard protocol to integrate third-party specialized services
Maintainability
Must continuously update virus signatures and detection rules
Professional vendors maintain the detection engine
1.2 Common Use Cases
File Upload Security: Documents uploaded to knowledge bases must undergo virus scanning first
Conversation Content Filtering: Detect whether conversations contain inappropriate language or sensitive information
Data Loss Prevention (DLP): Prevent users from uploading files containing confidential information
Compliance Checks: Ensure content complies with regulations such as GDPR and HIPAA
2. MaiAgent ICAP Integration Architecture
2.1 Core Component Overview
ICAP Client: MaiAgent's built-in ICAP protocol implementation responsible for communicating with the ICAP server
ICAP Server: Third-party content scanning service, such as antivirus gateways from Symantec, McAfee, or Trend Micro
Processing Queue: Asynchronously handles file uploads and scanning tasks to avoid blocking user operations
Scan Records: Stores all scanning results including file hash, scan time, results, and other information
2.2 ICAP Request Flow
3. ICAP Protocol Implementation Details
3.1 ICAP Request Format
MaiAgent uses REQMOD (Request Modification) mode to send files to the ICAP server:
Key field descriptions:
REQMOD: Request Modification mode, used to inspect content before storage
Encapsulated: Describes the encapsulation format of the HTTP request
Allow: 204: Informs the ICAP server that a 204 response is acceptable if no content modification is needed
Chunked Encoding: File content is sent using chunked transfer encoding
3.2 Chunked Transfer Encoding
MaiAgent correctly implements HTTP chunked transfer encoding:
Why is correct chunked encoding important?
The ICAP protocol requires HTTP chunked transfer encoding for sending request and response bodies
Incorrect encoding formats will cause the ICAP server to fail to parse file content correctly
MaiAgent has fixed the hexadecimal format representation of chunk sizes to ensure compatibility with the ICAP standard
3.3 ICAP Response Handling
The ICAP server may return the following responses:
200 OK
Content has been inspected or modified
Accept modified content (if any), store file
204 No Content
No content modification needed
Store the original file directly
403 Forbidden
Content is prohibited (e.g., contains a virus)
Reject storage, send ICAP_BLOCKED event to client
500 Internal Error
ICAP server error
Log error, send ICAP_ERROR event to client
4. WebChat Integration
When ICAP scanning is complete, the system notifies the user of the scan results in real time:
File Blocked: Displays a user-friendly error message explaining why the file was rejected (e.g., virus detected)
System Error: Prompts the user to try again later and logs the error
User Experience Optimization
Instant Feedback: Displays a "Scanning" status immediately after file upload
Progress Indicator: Shows a progress bar during large file uploads
Friendly Error Messages: Uses clear language to explain why a file was rejected
Retry Mechanism: Provides a re-upload option for temporary errors
5. Scan Record Management
5.1 Record Retention
MaiAgent stores detailed records of all ICAP scans:
Each scan record contains the following information:
File ID
Unique identifier of the scanned file
Filename
Original filename
File Hash
SHA-256 hash for verifying file integrity
Scan Result
PASS, BLOCKED, or ERROR
Scan Reason
Specific reason when a file is blocked
ICAP Server
Address of the ICAP server that performed the scan
Scan Time
Timestamp of when the scan was executed
User/Organization
User and organization that initiated the scan
5.2 Expired Record Cleanup
MaiAgent implements an automatic cleanup mechanism that periodically deletes expired scan records:
The system uses background scheduled tasks to automatically clean up scan records that exceed the retention period.
Cleanup strategy:
Retention Period: Scan records are retained for 90 days by default
Execution Frequency: Cleanup task runs once per week
Audit Retention: Critical security events (such as virus detections) can be configured with longer retention periods
6. Configuration and Deployment
6.1 ICAP Server Configuration
Administrators need to configure ICAP server information in the MaiAgent admin panel:
The following ICAP parameters can be configured in the admin panel:
ICAP Server Address
Connection URL for the ICAP service
Timeout
Scan timeout in seconds
Retry Count and Interval
Retry strategy for failed scans
Scan Scope
Choose to scan file uploads and/or conversation content
File Size Limit
Files exceeding this size will not be scanned
File Extension Whitelist
Specify file types that do not require scanning
6.2 Supported ICAP Services
MaiAgent is compatible with the following mainstream ICAP services:
Symantec Protection Engine: Enterprise-grade antivirus and content filtering
McAfee Web Gateway: Gateway-level content scanning
Trend Micro IWSVA: Integrated web security and content scanning
Kaspersky Scan Engine: Kaspersky's ICAP scanning engine
ClamAV (via c-icap): Open-source antivirus solution
6.3 Performance Considerations
To ensure system performance, the following recommendations apply:
Asynchronous Scanning: Use background tasks for scanning large files to avoid blocking upload responses
Caching: Cache scan results for identical files (same hash value)
ICAP Connection Pooling: Maintain persistent connections with the ICAP server to reduce connection overhead
Load Balancing: Use multiple ICAP server instances to distribute scanning load
7. Technical Advantages of MaiAgent's ICAP Integration
7.1 Security Advantages
Multi-layered Protection: Files are scanned before storage to prevent malicious content
Real-time Blocking: Threats are blocked immediately and never stored in the system
Professional Engines: Integrates industry-leading antivirus and DLP engines
Complete Records: All scanning activities are recorded in detail for security auditing
7.2 Compliance Advantages
Standard Protocol: Uses the industry-standard ICAP protocol for easy integration with existing enterprise security infrastructure
Audit Trail: Complete scan records meet compliance requirements
Flexible Configuration: Scanning strategies can be adjusted based on different industry compliance needs
Data Protection: Supports DLP engines to prevent sensitive data leakage
7.3 Operations Advantages
Decoupled Architecture: ICAP scanning logic is decoupled from the application, making maintenance easier
Horizontal Scaling: Easily add ICAP servers to handle more scan requests
Unified Management: Enterprises can use their existing ICAP infrastructure to centrally manage content scanning across all applications
Detailed Logging: Comprehensive scan logs for troubleshooting and performance optimization
8. Related Documentation
AI Safety Guardrails - Learn about MaiAgent's multi-layered security protection strategies
Reference Links
Last updated
Was this helpful?