API referenceKnowledge BaseCrawl jobs

Create crawl job

Create a crawl job to crawl the given URL with specified depth and page limits.

Headers

xi-api-keystringOptional

Request

This endpoint expects an object.
urlstringRequired
URL to a page of documentation that the agent will have access to in order to interact with users.
max_depthintegerOptional1-5Defaults to 3

Maximum depth for crawling (1-5), defaults to 3.

max_pagesintegerOptional1-10000Defaults to 1000

Maximum number of pages to crawl (1-10,000), defaults to 1000.

patternstringOptional<=2048 characters
If set, only URLs that match this pattern are included.
sitemap_urlslist of stringsOptional

List of URLs to crawl from sitemap (optional, overrides automatic URL discovery).

parent_folder_idstringOptional
If set, the created document or folder will be placed inside the given folder.
enable_auto_syncbooleanOptionalDefaults to false

Whether to enable auto-sync for this URL document.

auto_removebooleanOptionalDefaults to false

Whether to automatically remove the document if the URL becomes unavailable. Only applicable when auto-sync is enabled.

Response

Successful Response
idstring
typeenumDefaults to discovery
root_folder_idstring
statusstring
created_atinteger
folder_pathlist of objects
The folder path segments leading to the root folder, from root to parent folder.

Errors

422
Crawl Jobs Create Request Unprocessable Entity Error