# On the Suitability of LLM-Driven Agents for Dark Pattern Audits

arXiv: https://arxiv.org/abs/2603.03881 · HTML: https://mlsanity.com/paper/2603.03881/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 60 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [Dark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influence](https://mlsanity.com/paper/2001.02479/) (2020)
- [(Un)informed Consent: Studying GDPR Consent Notices in the Field](https://mlsanity.com/paper/1909.02638/) (2019)
- [Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs](https://mlsanity.com/paper/2404.17833/) (2024)
- [An Ontology of Dark Patterns Knowledge: Foundations, Definitions, and a Pathway for Shared Knowledge-Building](https://mlsanity.com/paper/2309.09640/) (2023)
- [AidUI: Toward Automated Recognition of Dark Patterns in User Interfaces](https://mlsanity.com/paper/2303.06782/) (2023)
- [Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications](https://mlsanity.com/paper/2308.05898/) (2023)
- [50 Shades of Deceptive Patterns: A Unified Taxonomy, Multimodal Detection, and Security Implications](https://mlsanity.com/paper/2501.13351/) (2025)
- [Getting Trapped in Amazon's "Iliad Flow": A Foundation for the Temporal Analysis of Dark Patterns](https://mlsanity.com/paper/2309.09635/) (2023)
- [Consumer Beware! Exploring Data Brokers' CCPA Compliance](https://mlsanity.com/paper/2506.21914/) (2025)
- [(Un)clear and (In)conspicuous: The right to opt-out of sale under CCPA](https://mlsanity.com/paper/2009.07884/) (2020)
- [Needle in the Haystack: Analyzing the Right of Access According to GDPR Article 15 Five Years after the Implementation](https://mlsanity.com/paper/2308.15166/) (2023)
- [Staying at the Roach Motel: Cross-Country Analysis of Manipulative Subscription and Cancellation Flows](https://mlsanity.com/paper/2309.17145/) (2023)
- [Dark Patterns in the Opt-Out Process and Compliance with the California Consumer Privacy Act (CCPA)](https://mlsanity.com/paper/2409.09222/) (2024)
- [C3PA: An Open Dataset of Expert-Annotated and Regulation-Aware Privacy Policies to Enable Scalable Regulatory Compliance Audits](https://mlsanity.com/paper/2410.03925/) (2024)

## Abstract (arXiv)

As LLM-driven agents begin to autonomously navigate the web, their ability to interpret and respond to manipulative interface design becomes critical. A fundamental question that emerges is: can such agents reliably recognize patterns of friction, misdirection, and coercion in interface design (i.e., dark patterns)? We study this question in a setting where the workflows are consequential: website portals associated with the submission of CCPA-related data rights requests. These portals operationalize statutory rights, but they are implemented as interactive interfaces whose design can be structured to facilitate, burden, or subtly discourage the exercise of those rights. We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows, structured evidence gathering, and classification of potential dark patterns. Across a set of 456 data broker websites, we evaluate: (1) the ability of the agent to consistently locate and complete request flows, (2) the reliability and reproducibility of its dark pattern classifications, and (3) the conditions under which it fails or produces poor judgments. Our findings characterize both the feasibility and the limitations of using LLM-driven agents for scalable dark pattern auditing.