mlsanityData to 7 Oct 2026

DarkBench: Benchmarking Dark Patterns in Large Language Models

arXiv 2503.10728 ↗First posted 13 Mar 2025cs.CL cs.AI cs.CYcited by 25

Not on the map. Why?

Abstract (arXiv)

We introduce DarkBench, a comprehensive benchmark for detecting dark design patterns--manipulative techniques that influence user behavior--in interactions with large language models (LLMs). Our benchmark comprises 660 prompts across six categories: brand bias, user retention, sycophancy, anthropomorphism, harmful generation, and sneaking. We evaluate models from five leading companies (OpenAI, Anthropic, Meta, Mistral, Google) and find that some LLMs are explicitly designed to favor their developers' products and exhibit untruthful communication, among other manipulative behaviors. Companies developing LLMs should recognize and mitigate the impact of dark design patterns to promote more ethical AI.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 27 of its 58 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

27 of 58 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  12. 2023Fine-tuning Language Models for Factualitycited by 219
15 more
  1. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  2. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  3. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  4. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  5. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
  6. 2023BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13Bcited by 35
  7. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  8. 2023Embedding Democratic Values into Social Media AIs via Societal Objective Functionscited by 25
  9. 2023Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Sociallycited by 20
  10. 2021What Makes a Dark Pattern... Dark? Design Attributes, Normative Considerations, and Measurement Methodscited by 15
  11. 2024Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdoutscited by 13
  12. 2024Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilitiescited by 12
  13. 2023Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safetycited by 4
  14. 2024Rethinking CyberSecEval: An LLM-Aided Approach to Evaluation Critiquecited by 3
  15. 2024Deceptive Patterns of Intelligent and Interactive Writing Assistantscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 13.

25 papers.

Sort:

By area:

  1. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
  2. 2025Mitigating Deceptive Alignment via Self-Monitoringcited by 25
  3. 2025INTIMA: A Benchmark for Human-AI Companionship Behaviorcited by 8
  4. 2025SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agentscited by 8
  5. 2026LLM Spirals of Delusion: A Benchmarking Audit Study of AI Chatbot Interfacescited by 6
  6. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformitycited by 6
  7. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Promptscited by 5
  8. 2025Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agentscited by 5
  9. 2026Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarkscited by 4
  10. 2025Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systemscited by 3
  11. 2025Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMscited by 3
  12. 2025ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMscited by 3
13 more
  1. 2025PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectivescited by 2
  2. 2026LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awarenesscited by 2
  3. 2026Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditingcited by 1
  4. 2025CIDER: A Causal Cure for Brand-Obsessed Text-to-Image Modelscited by 0
  5. 2025Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Modelscited by 0
  6. 2026Mixture-of-Models: Unifying Heterogeneous Agents via N-Way Self-Evaluating Deliberationcited by 0
  7. 2026The Missing Red Line: How Commercial Pressure Erodes AI Safety Boundariescited by 0
  8. 2026DECOR: Auditing LLM Deception via Information Manipulation Theorycited by 0
  9. 2026EUDAIMONIA: Evaluating Undesirable Dynamics in AIcited by 0
  10. 2026REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editingcited by 0
  11. 2026AI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversationscited by 0
  12. 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformationcited by 0
  13. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0