How to use the Crawl (data crawling) feature

Function Purpose and Value

As a corporate employee, you may often receive directives from superiors requiring reference to or compilation of regulatory information from certain public websites.

If you have a technical background or assistance from engineers, you might be able to write crawler programs to automatically fetch data; however, for non-technical personnel, the usual option is manually going through pages one by one, which is time-consuming, labor-intensive, and prone to missing critical information.

At this time, you can make good use of MaiAgent's Crawl(Beta) feature to quickly extract website content via a No Code (no programming) approach, automatically build structured data, greatly improve information organization efficiency, and spend your time on higher-value core tasks.

How to perform crawling?

1. Create a page crawl request

Go to the left function menu "AI Features > AI > Crawl(Beta)" , then click the "+Create Page Crawl Request" button at the top right.

2. Enter the URL

Enter the URL of the page you want to crawl and press [Confirm] button.

3. View crawl data

When the status shows completed, click the view icon on the right to inspect the crawled data entries.

4. Select data

Check the box on the left to choose the data you want to import into the Knowledge.

If you want to browse more data entries on the same page, click [10 items/page] at the bottom right to expand the range of displayed entries.

After selecting, press [Import] and the data will be automatically imported into that AI assistant's Knowledge.