# Identifying AI Web Scrapers Using Canary Tokens

arXiv: https://arxiv.org/abs/2605.13706 · HTML: https://mlsanity.com/paper/2605.13706/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 79 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 13 of 13)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [Consent in Crisis: The Rapid Decline of the AI Data Commons](https://mlsanity.com/paper/2407.14933/) (2024)
- [Somesite I Used To Crawl: Awareness, Agency and Efficacy in Protecting Content Creators From AI Crawlers](https://mlsanity.com/paper/2411.15091/) (2024)
- [Scrapers selectively respect robots.txt directives: evidence from a large-scale empirical study](https://mlsanity.com/paper/2505.21733/) (2025)
- [A Survey of Web Content Control for Generative AI](https://mlsanity.com/paper/2404.02309/) (2024)
- [A Whole New World: Creating a Parallel-Poisoned Web Only AI-Agents Can See](https://mlsanity.com/paper/2509.00124/) (2025)
- [Is Misinformation More Open? A Study of robots.txt Gatekeeping on the Web](https://mlsanity.com/paper/2510.10315/) (2025)
- [How Do Data Owners Say No? A Case Study of Data Consent Mechanisms in Web-Scraped Vision-Language AI Training Datasets](https://mlsanity.com/paper/2511.08637/) (2025)
- [Developer Experience with AI Coding Agents: HTTP Behavioral Signatures in Documentation Portals](https://mlsanity.com/paper/2604.02544/) (2026)

## Abstract (arXiv)

From pre-training to query-time augmentation, web-scraped data helps to improve the quality and contextual relevancy of content generated by large language models (LLMs). However, large-scale web scraping to feed LLMs can affect site stability and raise legal, privacy, or ethics concerns. If website owners wish to limit LLM-related web scraping on their site, due to these or other concerns, they may turn to scraper access control mechanisms like the Robots Exclusion Protocol. To be most effective, such mechanisms require site owners to first identify the scrapers that they wish to restrict (e.g., via User-Agent strings). Existing mechanisms to identify LLM-related scrapers rely on voluntary disclosure by companies, one-off experiments by researchers, or crowd-sourced reports -- methods that are neither reliable nor scalable. This paper proposes a novel technique for accurately and automatically inferring LLM-related scrapers. We host dynamic websites that serve unique canary tokens to each visiting scraper, then prompt LLMs for information about our sites. If an LLM consistently generates outputs containing tokens unique to a scraper, it provides evidence of exposure to that scraper. Via experiments across 22 production LLM systems, we demonstrate that our approach can reliably identify which scrapers feed which LLM, including several that are not publicly known or disclosed by the companies. Our approach provides a promising avenue for unprivileged third parties to infer which scrapers serve data to which LLMs, potentially enabling better control over unwanted scraping.