# Top-N Recommendation Algorithms: A Quest for the State-of-the-Art

arXiv: https://arxiv.org/abs/2203.01155 · HTML: https://mlsanity.com/paper/2203.01155/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2022.

## Built on by (most cited first, 12 of 12)

- [Infinite Recommendation Networks: A Data-Centric Approach](https://mlsanity.com/paper/2206.02626/) (2022)
- [Evaluating ChatGPT as a Recommender System: A Rigorous Approach](https://mlsanity.com/paper/2309.03613/) (2023)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [From Variability to Stability: Advancing RecSys Benchmarking Practices](https://mlsanity.com/paper/2402.09766/) (2024)
- [BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models](https://mlsanity.com/paper/2310.06278/) (2023)
- [How Fair is Your Diffusion Recommender Model?](https://mlsanity.com/paper/2409.04339/) (2024)
- [When Newer is Not Better: Does Deep Learning Really Benefit Recommendation From Implicit Feedback?](https://mlsanity.com/paper/2305.01801/) (2023)
- [Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis](https://mlsanity.com/paper/2308.00404/) (2023)
- [Post-hoc Selection of Pareto-Optimal Solutions in Search and Recommendation](https://mlsanity.com/paper/2306.12165/) (2023)
- [Performance Comparison of Session-based Recommendation Algorithms based on GNNs](https://mlsanity.com/paper/2312.16695/) (2023)
- [Job recommendations: benchmarking of collaborative filtering methods for classifieds](https://mlsanity.com/paper/2301.07946/) (2023)
- [Recommender Systems: A Primer](https://mlsanity.com/paper/2302.02579/) (2023)

## Abstract (arXiv)

Research on recommender systems algorithms, like other areas of applied machine learning, is largely dominated by efforts to improve the state-of-the-art, typically in terms of accuracy measures. Several recent research works however indicate that the reported improvements over the years sometimes "don't add up", and that methods that were published several years ago often outperform the latest models when evaluated independently. Different factors contribute to this phenomenon, including that some researchers probably often only fine-tune their own models but not the baselines. In this paper, we report the outcomes of an in-depth, systematic, and reproducible comparison of ten collaborative filtering algorithms - covering both traditional and neural models - on several common performance measures on three datasets which are frequently used for evaluation in the recent literature. Our results show that there is no consistent winner across datasets and metrics for the examined top-n recommendation task. Moreover, we find that for none of the accuracy measurements any of the considered neural models led to the best performance. Regarding the performance ranking of algorithms across the measurements, we found that linear models, nearest-neighbor methods, and traditional matrix factorization consistently perform well for the evaluated modest-sized, but commonly-used datasets. Our work shall therefore serve as a guideline for researchers regarding existing baselines to consider in future performance comparisons. Moreover, by providing a set of fine-tuned baseline models for different datasets, we hope that our work helps to establish a common understanding of the state-of-the-art for top-n recommendation tasks.