# The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization

arXiv: https://arxiv.org/abs/2202.11776 · HTML: https://mlsanity.com/paper/2202.11776/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2022.

## Built on by (most cited first, 29 of 29)

- [Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking](https://mlsanity.com/paper/2403.03185/) (2024)
- [AI Alignment with Changing and Influenceable Reward Functions](https://mlsanity.com/paper/2405.17713/) (2024)
- [What's In My Human Feedback? Learning Interpretable Descriptions of Preference Data](https://mlsanity.com/paper/2510.26202/) (2025)
- [Preference Dynamics Under Personalized Recommendations](https://mlsanity.com/paper/2205.13026/) (2022)
- [Embedding Democratic Values into Social Media AIs via Societal Objective Functions](https://mlsanity.com/paper/2307.13912/) (2023)
- [Direct Alignment with Heterogeneous Preferences](https://mlsanity.com/paper/2502.16320/) (2025)
- [Clickbait vs. Quality: How Engagement-Based Optimization Shapes the Content Landscape in Online Platforms](https://mlsanity.com/paper/2401.09804/) (2024)
- [Algorithmic Assistance with Recommendation-Dependent Preferences](https://mlsanity.com/paper/2208.07626/) (2022)
- [Auditing for Human Expertise](https://mlsanity.com/paper/2306.01646/) (2023)
- [Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints](https://mlsanity.com/paper/2306.13738/) (2023)
- [Accounting for AI and Users Shaping One Another: The Role of Mathematical Models](https://mlsanity.com/paper/2404.12366/) (2024)
- [Ad Insertion in LLM-Generated Responses](https://mlsanity.com/paper/2601.19435/) (2026)
- [Choosing the Right Weights: Balancing Value, Strategy, and Noise in Recommender Systems](https://mlsanity.com/paper/2305.17428/) (2023)
- [Intention Knowledge Graph Construction for User Intention Relation Modeling](https://mlsanity.com/paper/2412.11500/) (2024)
- [Impact of Decentralized Learning on Player Utilities in Stackelberg Games](https://mlsanity.com/paper/2403.00188/) (2024)
- [CTRL-Rec: Controlling Recommender Systems With Natural Language](https://mlsanity.com/paper/2510.12742/) (2025)
- [Learning to Suggest Breaks: Sustainable Optimization of Long-Term User Engagement](https://mlsanity.com/paper/2211.13585/) (2022)
- [Machine Love](https://mlsanity.com/paper/2302.09248/) (2023)
- [System-2 Recommenders: Disentangling Utility and Engagement in Recommendation Systems via Temporal Point-Processes](https://mlsanity.com/paper/2406.01611/) (2024)
- [LLM Agents Enable User-Governed Personalization Beyond Platform Boundaries](https://mlsanity.com/paper/2605.09794/) (2026)
- [Online SuBmodular + SuPermodular (BP) Maximization with Bandit Feedback](https://mlsanity.com/paper/2207.03091/) (2022)
- [The Burden of Interactive Alignment with Inconsistent Preferences](https://mlsanity.com/paper/2510.16368/) (2025)
- [Mind the Prompt: Self-adaptive Generation of Task Plan Explanations via LLMs](https://mlsanity.com/paper/2604.21092/) (2026)
- [Algorithmic Content Selection and the Impact of User Disengagement](https://mlsanity.com/paper/2410.13108/) (2024)
- [Evolutionary Prediction Games](https://mlsanity.com/paper/2503.03401/) (2025)
- [A Job I Like or a Job I Can Get: Designing Job Recommender Systems Using Field Experiments](https://mlsanity.com/paper/2603.21699/) (2026)
- [LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems](https://mlsanity.com/paper/2606.02883/) (2026)
- [Compass: Continuously Aligning Social Media Feeds via In-Situ Reflections](https://mlsanity.com/paper/2608.04274/) (2026)
- [Why Does Misinformation Propagate Faster? An Algorithmic Perspective on X](https://mlsanity.com/paper/2609.28947/) (2026)

## Abstract (arXiv)

Online platforms have a wealth of data, run countless experiments and use industrial-scale algorithms to optimize user experience. Despite this, many users seem to regret the time they spend on these platforms. One possible explanation is misaligned incentives: platforms are not optimizing for user happiness. We suggest the problem runs deeper, transcending the specific incentives of any particular platform, and instead stems from a mistaken foundational assumption: To understand what users want, platforms look at what users do. Yet research has demonstrated, and personal experience affirms, that we often make choices in the moment that are inconsistent with what we actually want. In this work, we develop a model of media consumption where users have inconsistent preferences. We consider a platform which simply wants to maximize user utility, but only observes user engagement. We show how our model of users' preference inconsistencies produces phenomena that are familiar from everyday experience, but difficult to capture in traditional user interaction models. A key ingredient in our model is a formulation for how platforms determine what to show users: they optimize over a large set of potential content (the content manifold) parametrized by underlying features of the content. Whether improving engagement improves user welfare depends on the direction of movement in the content manifold: for certain directions of change, increasing engagement makes users less happy, while in other directions, increasing engagement makes users happier. We characterize the structure of content manifolds for which increasing engagement fails to increase user utility. By linking these effects to abstractions of platform design choices, our model thus creates a theoretical framework and vocabulary in which to explore interactions between design, behavioral science, and social media.