ID Card Recognition

Use multimodal AI to recognize ID cards and other document images, automatically outputting structured JSON data

When enterprises process identity verification, account opening, onboarding, and other workflows, they often need to manually enter names, ID numbers, addresses, and other information from documents one by one — a process that is both time-consuming and error-prone. With MaiAgent's multimodal AI assistant, users simply upload a document photo, and the system automatically recognizes and outputs structured JSON data that can be directly integrated with downstream business systems.

Compared to traditional OCR solutions, MaiAgent combines the multimodal capabilities of large language models with the following advantages:

	Traditional OCR Engine	MaiAgent Multimodal AI
Layout Adaptability	Requires designing a template for each document type	Automatically understands different document layouts
Front/Back Processing	Requires separate processing and manual matching	Can recognize both front and back from a single image
Field Expansion	Requires code modifications	Simply modify the JSON Schema
Deployment Cost	Self-hosted OCR service or purchased license	Platform configuration, no coding required
Error Handling	Returns raw text, requires post-processing	AI understands semantics and automatically corrects formatting

Use Cases

• Banking Account Opening: Customer uploads ID card, automatically extracting name, ID number, and other information to fill in the account opening form

• HR Onboarding Process: New employees upload documents, and the system automatically creates records

• Medical Registration: Patients upload health insurance card or ID card, automatically filling in basic information

• Government Agency Counter Service: Citizens present documents for instant recognition and auto-population of application forms

Solution Architecture

Setup Steps

Step 1: Create an AI Assistant

1. Left sidebar → AI Features → AI Assistants

2. Click Create AI Assistant in the upper right corner

3. Fill in basic settings:

Field	Value
Assistant Name	Document OCR Recognition Assistant
Large Language Model	Gemini 3.1 Flash-Lite (must support multimodal)

Create AI Assistant basic settings

When selecting a large language model, make sure the model supports multimodal (image input), such as the Gemini series. If you select a text-only model, it will not be able to recognize image content.

Step 2: Set Response Mode and JSON Schema

Switch to the Response Mode Settings tab:

Field	Value
Response Mode	RAG Q&A
Output Format	Output JSON Format

After selecting Output JSON Format, enter the following content in the JSON Schema field below:

{
  "type": "object",
  "properties": {
    "document_type": {
      "type": "string",
      "description": "Document type, such as ROC National ID Card, driver's license, passport, health insurance card, residence permit, etc."
    },
    "name": {
      "type": "string",
      "description": "Full name of the document holder, read from the name field on the front"
    },
    "gender": {
      "type": "string",
      "enum": ["Male", "Female"],
      "description": "Gender"
    },
    "date_of_birth": {
      "type": "string",
      "description": "Date of birth, in ROC calendar year format"
    },
    "military_service": {
      "type": ["string", "null"],
      "description": "Military service status, such as exempt, completed, pending, alternative service. Null for females or if field is absent"
    },
    "place_of_birth": {
      "type": ["string", "null"],
      "description": "Place of birth, read from the back"
    },
    "id_number": {
      "type": "string",
      "description": "National ID number, 1 English letter + 9 digits"
    },
    "expiry_or_issue_date": {
      "type": ["string", "null"],
      "description": "Issue date or expiry date, in ROC calendar year format"
    },
    "address": {
      "type": ["string", "null"],
      "description": "Full address, read from the address field on the back, including city, district, borough, neighborhood, road, section, lane, alley, number, and floor"
    }
  },
  "required": [
    "document_type", "name", "gender", "date_of_birth",
    "military_service", "place_of_birth", "id_number",
    "expiry_or_issue_date", "address"
  ]
}

JSON Schema output format settings

Step 3: Write Role Instructions

In the Role Instructions section on the same tab, enter the following content:

You are a professional document OCR recognition assistant. Users will upload document images, and you must carefully recognize all text in the image and extract key information.

## Most Important: Carefully Scan All Areas of the Entire Image

⚠️ Users often photograph both the front and back of a document in the same image (side by side or stacked vertically). You must:
1. First scan the entire image to identify how many document faces are present (may be 1 or 2)
2. Separately recognize the text in each area
3. Merge the front and back information into the results

Never say "back not provided" or "information not in the image" unless you have confirmed that the information truly cannot be found in any area of the entire image.

## ROC National ID Card Field Locations

### Front (has national flag, photo)
- "Name": Next to the photo, this is the document holder's own name
- "Date of Birth": ROC Year XX Month XX Day XX
- "Gender": Marked as "Male" or "Female"
- "ID Number": At the bottom, 1 uppercase English letter + 9 digits
- "Issue Date": Bottom left

### Back (has father/mother/spouse fields, background is usually darker)
- "Father" "Mother": Parents' names (not the document holder)
- "Spouse": Spouse's name
- "Military Service": Military service status
- "Place of Birth": e.g., "Taipei City", "New Taipei City"
- "Address": Full registered address

## Recognition Rules
1. Document Type: Determine based on appearance
2. Name: Must be read from the "Name" field on the front; do not read the father/mother/spouse names from the back
3. Gender: Read from the front
4. Date of Birth: Read from the front, in ROC calendar year format
5. Military Service: Read from the back. Females typically have no military service; fill null
6. Place of Birth: Read from the back
7. ID Number: Read from the bottom of the front, carefully recognizing each character
8. Issue Date: Read from the "Issue Date" field on the front
9. Address: Read from the "Address" field on the back, completely including city, district, borough, neighborhood, road, section, lane, alley, number, and floor

## Output Notes
- Fill null for fields that cannot be recognized
- All recognition results must be faithful to the image content; do not guess or fabricate

Role instructions settings

Click Save to complete the assistant setup.

Step 4: Test Recognition Results

1. Enter the assistant's Test Conversation

2. Upload a document image (can include both front and back)

3. Enter "Please recognize this document image"

4. Verify the returned JSON structure and field content are correct

Document recognition test results

Usage Example

Upload an image containing both the front and back of an ID card, and the AI assistant will automatically recognize and output structured JSON:

{
  "document_type": "ROC National ID Card",
  "name": "陳筱玲",
  "gender": "Female",
  "date_of_birth": "ROC Year 57, June 5",
  "military_service": null,
  "place_of_birth": "Taipei City",
  "id_number": "A234567890",
  "expiry_or_issue_date": "ROC Year 94, July 1",
  "address": "No. 218, Aly. 165, Ln. 283, Sec. 6, Minquan E. Rd., Huzhou Borough, Neihu Dist., Taipei City"
}

Recognition accuracy depends on image quality. Ensure adequate lighting when taking photos, avoid glare, and lay the document flat for photography. If photographing both front and back simultaneously, arrange them side by side or stacked vertically.

JSON Output Field Descriptions

Field	Type	Source	Description
document_type	string	Front	Document type
name	string	Front	Document holder's name
gender	string	Front	Gender (Male/Female)
date_of_birth	string	Front	Date of birth (ROC calendar year)
military_service	string/null	Back	Military service status
place_of_birth	string/null	Back	Place of birth
id_number	string	Front	National ID number
expiry_or_issue_date	string/null	Front	Issue date
address	string/null	Back	Full address

API Integration

This assistant can be integrated with business systems via the MaiAgent API to automate the document recognition process.

Call Flow

Step 1: Upload Image to Get Attachment ID

curl -X POST \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -F "file=@/path/to/id_card.jpeg" \
  "https://api.maiagent.ai/api/v1/attachments-upload/"

Note the id, type, file, and filename fields from the response — they will be used in the next step.

Step 2: Call the Assistant for Recognition

curl -X POST \
  -H "Authorization: Api-Key YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "message": {
      "content": "Please recognize this document image",
      "attachments": [{
        "id": "{id from Step 1}",
        "type": "image",
        "file": "{file URL from Step 1}",
        "filename": "id_card.jpeg"
      }]
    },
    "isStreaming": false
  }' \
  "https://api.maiagent.ai/api/v1/chatbots/{assistant_id}/completions/"

Step 3: Parse the Response

The content field in the response contains the recognition result in JSON format:

Python

import json
ocr_result = json.loads(response["content"])
print(ocr_result["name"])             # 陳筱玲
print(ocr_result["id_number"])        # A234567890
print(ocr_result["address"])          # 臺北市內湖區葫洲里1鄰民權東路六段283巷165弄218號

JavaScript

const ocrResult = JSON.parse(response.content);
console.log(ocrResult.name);             // 陳筱玲
console.log(ocrResult.id_number);        // A234567890
console.log(ocrResult.address);          // 臺北市內湖區葫洲里1鄰民權東路六段283巷165弄218號

For complete example code, refer to Playma-Co-Ltd/maiagent-api-examples.