EvalRS: a Rounded Evaluation of Recommender Systems
Not on the map. Why?
Abstract (arXiv)
Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings while giving little attention to the evaluation of RS behavior in real-world scenarios. Such narrow focus has limited the capacity of RSs to have a lasting impact in the real world and makes them vulnerable to undesired behavior, such as reinforcing data biases. We propose EvalRS as a new type of challenge, in order to foster this discussion among practitioners and build in the open new methodologies for testing RSs "in the wild".
Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). First posted on arXiv 12 Jul 2022.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2022: 3; 2023: 1.
4 papers.
- 2023Reasonable Scale Machine Learning with Open-Source Metaflowcited by 5
- 2022Triplet Losses-based Matrix Factorization for Robust Recommendationscited by 0
- 2022Track2Vec: fairness music recommendation with a GPU-free customizable-driven frameworkcited by 0
- 2022Item-based Variational Auto-encoder for Fair Music Recommendationcited by 0
- 2023Reasonable Scale Machine Learning with Open-Source Metaflow21 Mar 2023
- 2022Track2Vec: fairness music recommendation with a GPU-free customizable-driven framework29 Oct 2022
- 2022Item-based Variational Auto-encoder for Fair Music Recommendation24 Oct 2022
- 2022Triplet Losses-based Matrix Factorization for Robust Recommendations21 Oct 2022