# IllusionCAPTCHA: A CAPTCHA based on Visual Illusion

arXiv: https://arxiv.org/abs/2502.05461 · HTML: https://mlsanity.com/paper/2502.05461/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2025.

## Built on by (most cited first, 13 of 13)

- [MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attacks](https://mlsanity.com/paper/2506.05982/) (2025)
- [Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions](https://mlsanity.com/paper/2507.22617/) (2025)
- [COGNITION: From Evaluation to Defense against Multimodal LLM CAPTCHA Solvers](https://mlsanity.com/paper/2512.02318/) (2025)
- [Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation](https://mlsanity.com/paper/2510.03863/) (2025)
- [Next-Gen CAPTCHAs: Leveraging the Cognitive Gap for Scalable and Diverse GUI-Agent Defense](https://mlsanity.com/paper/2602.09012/) (2026)
- [Broken Gates: Re-evaluating Web Bot Defenses in the Age of LLM Agents](https://mlsanity.com/paper/2607.18659/) (2026)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [Beyond BeautifulSoup: Benchmarking LLM-Powered Web Scraping for Everyday Users](https://mlsanity.com/paper/2601.06301/) (2026)
- [VIPER Strike: Defeating Visual Reasoning CAPTCHAs via Structured Vision-Language Inference](https://mlsanity.com/paper/2601.06461/) (2026)
- [HLL: Can Agents Cross Humanity's Last Line of Verification?](https://mlsanity.com/paper/2606.02449/) (2026)
- [Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions](https://mlsanity.com/paper/2607.19061/) (2026)
- [When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI](https://mlsanity.com/paper/2512.03087/) (2025)
- [Artificial Effort](https://mlsanity.com/paper/2605.23920/) (2026)

## Abstract (arXiv)

CAPTCHAs have long been essential tools for protecting applications from automated bots. Initially designed as simple questions to distinguish humans from bots, they have become increasingly complex to keep pace with the proliferation of CAPTCHA-cracking techniques employed by malicious actors. However, with the advent of advanced large language models (LLMs), the effectiveness of existing CAPTCHAs is now being undermined. To address this issue, we have conducted an empirical study to evaluate the performance of multimodal LLMs in solving CAPTCHAs and to assess how many attempts human users typically need to pass them. Our findings reveal that while LLMs can solve most CAPTCHAs, they struggle with those requiring complex reasoning type of CAPTCHA that also presents significant challenges for human users. Interestingly, our user study shows that the majority of human participants require a second attempt to pass these reasoning CAPTCHAs, a finding not reported in previous research. Based on empirical findings, we present IllusionCAPTCHA, a novel security mechanism employing the "Human-Easy but AI-Hard" paradigm. This new CAPTCHA employs visual illusions to create tasks that are intuitive for humans but highly confusing for AI models. Furthermore, we developed a structured, step-by-step method that generates misleading options, which particularly guide LLMs towards making incorrect choices and reduce their chances of successfully solving CAPTCHAs. Our evaluation shows that IllusionCAPTCHA can effectively deceive LLMs 100% of the time. Moreover, our structured design significantly increases the likelihood of AI errors when attempting to solve these challenges. Results from our user study indicate that 86.95% of participants successfully passed the CAPTCHA on their first attempt, outperforming other CAPTCHA systems.