# Characterizing and modeling harms from interactions with design patterns in AI interfaces

arXiv: https://arxiv.org/abs/2404.11370 · HTML: https://mlsanity.com/paper/2404.11370/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 123 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [Evaluating the Social Impact of Generative AI Systems in Systems and Society](https://mlsanity.com/paper/2306.05949/) (2023)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Auditing Radicalization Pathways on YouTube](https://mlsanity.com/paper/1908.08313/) (2019)
- [Designing for Responsible Trust in AI Systems: A Communication Perspective](https://mlsanity.com/paper/2204.13828/) (2022)
- [What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents](https://mlsanity.com/paper/1901.06525/) (2019)
- [Mirages: On Anthropomorphism in Dialogue Systems](https://mlsanity.com/paper/2305.09800/) (2023)
- [Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially](https://mlsanity.com/paper/2302.04720/) (2023)
- [What Makes a Dark Pattern... Dark? Design Attributes, Normative Considerations, and Measurement Methods](https://mlsanity.com/paper/2101.04843/) (2021)
- [Designing for Human Rights in AI](https://mlsanity.com/paper/2005.04949/) (2020)
- [The Contestation of Tech Ethics: A Sociotechnical Approach to Technology Ethics in Practice](https://mlsanity.com/paper/2106.01784/) (2021)
- [Circumvention by design -- dark patterns in cookie consents for online news outlets](https://mlsanity.com/paper/2006.13985/) (2020)
- [Dark Patterns and the Legal Requirements of Consent Banners: An Interaction Criticism Perspective](https://mlsanity.com/paper/2009.10194/) (2020)
- [End User Accounts of Dark Patterns as Felt Manipulation](https://mlsanity.com/paper/2010.11046/) (2020)
- [Dark Patterns in Online Shopping: Of Sneaky Tricks, Perceived Annoyance and Respective Brand Trust](https://mlsanity.com/paper/2107.07893/) (2021)

## Built on by (most cited first, 11 of 11)

- [Towards interactive evaluations for interaction harms in human-AI systems](https://mlsanity.com/paper/2405.10632/) (2024)
- [Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Models](https://mlsanity.com/paper/2502.07077/) (2025)
- [Thinking beyond the anthropomorphic paradigm benefits LLM research](https://mlsanity.com/paper/2502.09192/) (2025)
- [A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies](https://mlsanity.com/paper/2502.09870/) (2025)
- [A Pragmatic View of AI Personhood](https://mlsanity.com/paper/2510.26396/) (2025)
- [Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems](https://mlsanity.com/paper/2502.14019/) (2025)
- ["I Am the One and Only, Your Cyber BFF": Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI](https://mlsanity.com/paper/2410.08526/) (2024)
- [Ambiguity Collapse by LLMs: A Taxonomy of Epistemic Risks](https://mlsanity.com/paper/2603.05801/) (2026)
- [Imposing AI: Deceptive design patterns against sustainability](https://mlsanity.com/paper/2508.08672/) (2025)
- [With a Little Help From My Friends: Collective Manipulation in Risk-Controlling Recommender Systems](https://mlsanity.com/paper/2603.28476/) (2026)
- [Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users' Perspectives on Opportunities, Risks, and Mitigation Strategies](https://mlsanity.com/paper/2602.08187/) (2026)

## Abstract (arXiv)

The proliferation of applications using artificial intelligence (AI) systems has led to a growing number of users interacting with these systems through sophisticated interfaces. Human-computer interaction research has long shown that interfaces shape both user behavior and user perception of technical capabilities and risks. Yet, practitioners and researchers evaluating the social and ethical risks of AI systems tend to overlook the impact of anthropomorphic, deceptive, and immersive interfaces on human-AI interactions. Here, we argue that design features of interfaces with adaptive AI systems can have cascading impacts, driven by feedback loops, which extend beyond those previously considered. We first conduct a scoping review of AI interface designs and their negative impact to extract salient themes of potentially harmful design patterns in AI interfaces. Then, we propose Design-Enhanced Control of AI systems (DECAI), a conceptual model to structure and facilitate impact assessments of AI interface designs. DECAI draws on principles from control systems theory -- a theory for the analysis and design of dynamic physical systems -- to dissect the role of the interface in human-AI systems. Through two case studies on recommendation systems and conversational language model systems, we show how DECAI can be used to evaluate AI interface designs.