mlsanityData to 7 Oct 2026

Accounting for AI and Users Shaping One Another: The Role of Mathematical Models

arXiv 2404.12366 ↗First posted 18 Apr 2024cs.LG cs.CY cs.GTcited by 11

Not on the map. Why?

Abstract (arXiv)

As AI systems enter into a growing number of societal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of AI systems rarely accounts for how AI and users shape one another. In this position paper, we argue for the development of formal interaction models which mathematically specify how AI and users shape one another. Formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate societal impacts via counterfactual analysis, and (4) control societal impacts via interventions. The design space of formal interaction models is vast, and model design requires careful consideration of factors such as style, granularity, mathematical complexity, and measurability. Using content recommender systems as a case study, we critically examine the nascent literature of formal interaction models with respect to these use-cases and design axes. More broadly, we call for the community to leverage formal interaction models when designing, evaluating, or auditing any AI system which interacts with users.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 60 of its 94 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

60 of 94 references.

References · 60

  1. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  2. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  3. 2020Performative Predictioncited by 318
  4. 2019Measurement and Fairnesscited by 299
  5. 2015Strategic Classificationcited by 256
  6. 2018Delayed Impact of Fair Machine Learningcited by 183
  7. 2017How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utilitycited by 116
  8. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 91
  9. 2020Feedback Loop and Bias Amplification in Recommender Systemscited by 89
  10. 2018How Do Classifiers Induce Agents To Invest Effort Strategically?cited by 82
  11. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  12. 2019Degenerate Feedback Loops in Recommender Systemscited by 67
38 more
  1. 2018RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertisingcited by 65
  2. 2017Rotting Banditscited by 57
  3. 2015Bayesian Incentive-Compatible Bandit Explorationcited by 53
  4. 2019Roles for Computing in Social Changecited by 49
  5. 2021Alternative Microfoundations for Strategic Classificationcited by 47
  6. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
  7. 2022CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender Systemcited by 38
  8. 2022Modeling Content Creator Incentives on Algorithm-Curated Platformscited by 35
  9. 2018A Game-Theoretic Approach to Recommendation Systems with Strategic Content Providerscited by 33
  10. 2022Supply-Side Equilibria in Recommender Systemscited by 33
  11. 2020Optimizing Long-term Social Welfare in Recommender Systems: A Constrained Matching Approachcited by 32
  12. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  13. 2019Recovering Banditscited by 28
  14. 2022The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimizationcited by 28
  15. 2023How Bad is Top-K Recommendation under Competing Content Creators?cited by 27
  16. 2019Stochastic Bandits with Delay-Dependent Payoffscited by 25
  17. 2020Do Offline Metrics Predict Online Performance in Recommender Systems?cited by 25
  18. 2022Preference Dynamics Under Personalized Recommendationscited by 25
  19. 2021User Tampering in Reinforcement Learning Recommender Systemscited by 23
  20. 2020From Optimizing Engagement to Measuring Valuecited by 21
  21. 2021RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystemscited by 21
  22. 2020Manipulation-Proof Machine Learningcited by 19
  23. 2020Rebounding Bandits for Modeling Satiation Effectscited by 19
  24. 2022Reward Reports for Reinforcement Learningcited by 18
  25. 2023Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Modelscited by 17
  26. 2023Improved Bayes Risk Can Yield Reduced Social Welfare Under Competitioncited by 14
  27. 2024Clickbait vs. Quality: How Engagement-Based Optimization Shapes the Content Landscape in Online Platformscited by 13
  28. 2022Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systemscited by 12
  29. 2023Performative Recommendation: Diversifying Content via Strategic Incentivescited by 12
  30. 2023A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systemscited by 12
  31. 2022Modeling Attrition in Recommender Systems with Departing Banditscited by 11
  32. 2022YouTube, The Great Radicalizer? Auditing and Mitigating Ideological Biases in YouTube Recommendationscited by 10
  33. 2023Rethinking Incentives in Recommender Systems: Are Monotone Rewards Always Beneficial?cited by 10
  34. 2022Diversified Recommendations for Agents with Adaptive Preferencescited by 9
  35. 2021A Last Switch Dependent Analysis of Satiation and Seasonality in Banditscited by 8
  36. 2022Towards Psychologically-Grounded Dynamic Preference Modelscited by 8
  37. 2023Learning with Exposure Constraints in Recommendation Systemscited by 8
  38. 2023Recommending to Strategic Userscited by 8

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 9; 2026: 1.

11 papers.

  1. 2025Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Facecited by 12
  2. 2025AI-Assisted Decision Making with Human Learningcited by 6
  3. 2025Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamicscited by 5
  4. 2024Strategic Linear Contextual Banditscited by 4
  5. 2025The Backfiring Effect of Weak AI Safety Regulationcited by 3
  6. 2025Collective dynamics of strategic classificationcited by 1
  7. 2025Collaborating with GenAI: Incentives and Replacementscited by 1
  8. 2025The Burden of Interactive Alignment with Inconsistent Preferencescited by 1
  9. 2025Evolutionary Prediction Gamescited by 0
  10. 2025Data Sharing with a Generative AI Competitorcited by 0
  11. 2026Socially-Aware Recommender Systems Mitigate Opinion Clusterizationcited by 0