# BARS: Towards Open Benchmarking for Recommender Systems

arXiv: https://arxiv.org/abs/2205.09626 · HTML: https://mlsanity.com/paper/2205.09626/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2022.

## Built on by (most cited first, 30 of 54)

- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [On the Embedding Collapse when Scaling up Recommendation Models](https://mlsanity.com/paper/2310.04400/) (2023)
- [STEM: Unleashing the Power of Embeddings for Multi-task Recommendation](https://mlsanity.com/paper/2308.13537/) (2023)
- [Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems](https://mlsanity.com/paper/2305.12102/) (2023)
- [Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation](https://mlsanity.com/paper/2311.18213/) (2023)
- [External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation](https://mlsanity.com/paper/2502.17494/) (2025)
- [From Variability to Stability: Advancing RecSys Benchmarking Practices](https://mlsanity.com/paper/2402.09766/) (2024)
- [EB-NeRD: A Large-Scale Dataset for News Recommendation](https://mlsanity.com/paper/2410.03432/) (2024)
- [ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance](https://mlsanity.com/paper/2312.08852/) (2023)
- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [Massive Memorization with Hundreds of Trillions of Parameters for Sequential Transducer Generative Recommenders](https://mlsanity.com/paper/2510.22049/) (2025)
- [InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction](https://mlsanity.com/paper/2411.09852/) (2024)
- [LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models](https://mlsanity.com/paper/2408.01460/) (2024)
- [Recommendation Unlearning via Matrix Correction](https://mlsanity.com/paper/2307.15960/) (2023)
- [ToolSpectrum : Towards Personalized Tool Utilization for Large Language Models](https://mlsanity.com/paper/2505.13176/) (2025)
- [A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems](https://mlsanity.com/paper/2406.17335/) (2024)
- [Field Matters: A Lightweight LLM-enhanced Method for CTR Prediction](https://mlsanity.com/paper/2505.14057/) (2025)
- [PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents](https://mlsanity.com/paper/2605.07039/) (2026)
- [Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis](https://mlsanity.com/paper/2308.00404/) (2023)
- [Revisiting Feature Interactions from the Perspective of Quadratic Neural Networks for Click-through Rate Prediction](https://mlsanity.com/paper/2505.17999/) (2025)
- [ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation](https://mlsanity.com/paper/2606.28357/) (2026)
- [On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective](https://mlsanity.com/paper/2411.13052/) (2024)
- [CAdam: Confidence-Based Optimization for Online Learning](https://mlsanity.com/paper/2411.19647/) (2024)
- [RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation](https://mlsanity.com/paper/2509.03131/) (2025)
- [Adaptive Regularization for Large-Scale Sparse Feature Embedding Models](https://mlsanity.com/paper/2511.06374/) (2025)
- [Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction](https://mlsanity.com/paper/2601.19142/) (2026)
- [Weighted Multi-Level Feature Factorization for App ads CTR and installation prediction](https://mlsanity.com/paper/2308.02568/) (2023)
- [RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction](https://mlsanity.com/paper/2404.02249/) (2024)
- [Beyond KAN: Introducing KarSein for Adaptive High-Order Feature Interaction Modeling in CTR Prediction](https://mlsanity.com/paper/2408.08713/) (2024)

## Abstract (arXiv)

The past two decades have witnessed the rapid development of personalized recommendation techniques. Despite significant progress made in both research and practice of recommender systems, to date, there is a lack of a widely-recognized benchmarking standard in this field. Many existing studies perform model evaluations and comparisons in an ad-hoc manner, for example, by employing their own private data splits or using different experimental settings. Such conventions not only increase the difficulty in reproducing existing studies, but also lead to inconsistent experimental results among them. This largely limits the credibility and practical value of research results in this field. To tackle these issues, we present an initiative project (namely BARS) aiming for open benchmarking for recommender systems. In comparison to some earlier attempts towards this goal, we take a further step by setting up a standardized benchmarking pipeline for reproducible research, which integrates all the details about datasets, source code, hyper-parameter settings, running logs, and evaluation results. The benchmark is designed with comprehensiveness and sustainability in mind. It covers both matching and ranking tasks, and also enables researchers to easily follow and contribute to the research in this field. This project will not only reduce the redundant efforts of researchers to re-implement or re-run existing baselines, but also drive more solid and reproducible research on recommender systems. We would like to call upon everyone to use the BARS benchmark for future evaluation, and contribute to the project through the portal at: https://openbenchmark.github.io/BARS.