mlsanityData to 14 Oct 2026

We're Still Doing It (All) Wrong: Recommender Systems, Fifteen Years Later

arXiv 2509.09414 ↗First posted 11 Sep 2025cs.IR cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

In 2011, Xavier Amatriain sounded the alarm: recommender systems research was "doing it all wrong" [1]. His critique, rooted in statistical misinterpretation and methodological shortcuts, remains as relevant today as it was then. But rather than correcting course, we added new layers of sophistication on top of the same broken foundations. This paper revisits Amatriain's diagnosis and argues that many of the conceptual, epistemological, and infrastructural failures he identified still persist, in more subtle or systemic forms. Drawing on recent work in reproducibility, evaluation methodology, environmental impact, and participatory design, we showcase how the field's accelerating complexity has outpaced its introspection. We highlight ongoing community-led initiatives that attempt to shift the paradigm, including workshops, evaluation frameworks, and calls for value-sensitive and participatory research. At the same time, we contend that meaningful change will require not only new metrics or better tooling, but a fundamental reframing of what recommender systems research is for, who it serves, and how knowledge is produced and validated. Our call is not just for technical reform, but for a recommender systems research agenda grounded in epistemic humility, human impact, and sustainable practice.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 59 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

22 of 59 references.

References · 22

  1. 2017How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utilitycited by 120
  2. 2018Troubling Trends in Machine Learning Scholarshipcited by 108
  3. 2018Sequence-Aware Recommender Systemscited by 58
  4. 2020Exploring Data Splitting Strategies for the Evaluation of Recommendation Modelscited by 48
  5. 2021Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffscited by 48
  6. 2021Evaluating the Robustness of Off-Policy Evaluationcited by 32
  7. 2022Building Human Values into Recommender Systems: An Interdisciplinary Synthesiscited by 22
  8. 2023Widespread Flaws in Offline Evaluation of Recommender Systemscited by 22
  9. 2022Trustworthy Recommender Systemscited by 21
  10. 2021T-RECS: A Simulation Tool to Study the Societal Impact of Recommender Systemscited by 12
  11. 2025Do LLMs Memorize Recommendation Datasets? A Preliminary Study on MovieLens-1Mcited by 12
  12. 2024From Clicks to Carbon: The Environmental Toll of Recommender Systemscited by 11
10 more
  1. 2024Recommender Systems for Sustainability: Overview and Research Issuescited by 5
  2. 2024Search and Society: Reimagining Information Access for Radical Futurescited by 4
  3. 2024Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevancecited by 4
  4. 2025De-centering the (Traditional) User: Multistakeholder Evaluation of Recommender Systemscited by 4
  5. 2025Recommending With, Not For: Co-Designing Recommender Systems for Social Goodcited by 4
  6. 2023Distributionally-Informed Recommender System Evaluationcited by 2
  7. 2024Evaluating the performance-deviation of itemKNN in RecBole and LensKitcited by 2
  8. 2024Recommender Systems for Good (RS4Good): Survey of Use Cases and a Call to Action for Research that Matterscited by 2
  9. 2025Recommender Systems for Social Good: The Role of Accountability and Sustainabilitycited by 2
  10. 2025Extending MovieLens-32M to Provide New Evaluation Objectivescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.