# Beyond RMSE and MAE: Introducing EAUC to unmask hidden bias and unfairness in dyadic regression models

arXiv: https://arxiv.org/abs/2401.10690 · HTML: https://mlsanity.com/paper/2401.10690/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 52 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Compositional Fairness Constraints for Graph Embeddings](https://mlsanity.com/paper/1905.10674/) (2019)
- [Neural Collaborative Filtering vs. Matrix Factorization Revisited](https://mlsanity.com/paper/2005.09683/) (2020)
- [A Survey on the Fairness of Recommender Systems](https://mlsanity.com/paper/2206.03761/) (2022)
- [Beyond Parity: Fairness Objectives for Collaborative Filtering](https://mlsanity.com/paper/1705.08804/) (2017)
- [Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks](https://mlsanity.com/paper/1704.06803/) (2017)
- [Exploring Data Splitting Strategies for the Evaluation of Recommendation Models](https://mlsanity.com/paper/2007.13237/) (2020)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [Understanding Echo Chambers in E-commerce Recommender Systems](https://mlsanity.com/paper/2007.02474/) (2020)
- [What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective](https://mlsanity.com/paper/2206.04101/) (2022)
- [Filter Bubbles in Recommender Systems: Fact or Fallacy -- A Systematic Review](https://mlsanity.com/paper/2307.01221/) (2023)
- [Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks](https://mlsanity.com/paper/1811.01910/) (2018)
- [GLocal-K: Global and Local Kernels for Recommender Systems](https://mlsanity.com/paper/2108.12184/) (2021)
- [Echo Chambers in Collaborative Filtering Based Recommendation Systems](https://mlsanity.com/paper/2011.03890/) (2020)
- [Geometric Matrix Completion via Sylvester Multi-Graph Neural Network](https://mlsanity.com/paper/2206.09477/) (2022)
- [Prediction in complex systems: the case of the international trade network](https://mlsanity.com/paper/1511.05404/) (2015)

## Abstract (arXiv)

Dyadic regression models, which output real-valued predictions for pairs of entities, are fundamental in many domains (e.g. obtaining user-product ratings in Recommender Systems) and promising and under exploration in others (e.g. tuning patient-drug dosages in precision pharmacology). In this work, we prove that non-uniform observed value distributions of individual entities lead to severe biases in state-of-the-art models, skewing predictions towards the average of observed past values for the entity and providing worse-than-random predictive power in eccentric yet crucial cases; we name this phenomenon eccentricity bias. We show that global error metrics like Root Mean Squared Error (RMSE) are insufficient to capture this bias, and we introduce Eccentricity-Area Under the Curve (EAUC) as a novel metric that can quantify it in all studied domains and models. We prove the intuitive interpretation of EAUC by experimenting with naive post-training bias corrections, and theorize other options to use EAUC to guide the construction of fair models. This work contributes a bias-aware evaluation of dyadic regression to prevent unfairness in critical real-world applications of such systems.