> This is a page from the ElevenLabs documentation. For a complete page index, fetch https://elevenlabs.io/docs/llms.txt. For the full documentation in a single file, fetch https://elevenlabs.io/docs/llms-full.txt.

# ElevenLabs bots

ElevenLabs uses web crawlers to collect publicly available content for its AI models and to fetch pages for its products. Site owners can control their access with the user agent tokens below in `robots.txt`. Each setting is independent of the other: for example, a site can allow `ElevenlabsBot` so that ElevenLabs agents can use its pages, while disallowing `ElevenlabsCrawler`. Changes to `robots.txt` are reflected within about 24 hours.

| User agent          | Description & details                                                                                                                                                                                                                                                                                                                                                          |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ElevenlabsCrawler` | `ElevenlabsCrawler` crawls the web to collect content that may be used to train ElevenLabs' generative AI models. Disallowing `ElevenlabsCrawler` in `robots.txt` indicates that a site's content should not be used for training. Full user agent string: `Mozilla/5.0 (compatible; ElevenlabsCrawler/1.0; +https://elevenlabs.io/docs/bots)`. The version number may change. |
| `ElevenlabsBot`     | `ElevenlabsBot` fetches pages that ElevenLabs customers add to their agents' knowledge bases, when they add them and when they refresh them. It honors `robots.txt`. Full user agent string: `ElevenlabsBot/1.0`. The version number may change.                                                                                                                               |