# Fairmetrics: An R package for group fairness evaluation

arXiv: https://arxiv.org/abs/2506.06243 · HTML: https://mlsanity.com/paper/2506.06243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 7 of 7)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [De-biasing "bias" measurement](https://mlsanity.com/paper/2205.05770/) (2022)
- [fairmodels: A Flexible Tool For Bias Detection, Visualization, And Mitigation](https://mlsanity.com/paper/2104.00507/) (2021)
- [What is Fair? Defining Fairness in Machine Learning for Health](https://mlsanity.com/paper/2406.09307/) (2024)

## Built on by (most cited first, 1 of 1)

- [Nishpaksh: TEC Standard-Compliant Framework for Fairness Auditing and Certification of AI Models](https://mlsanity.com/paper/2601.16926/) (2026)

## Abstract (arXiv)

Fairness is a growing area of machine learning (ML) that focuses on ensuring models do not produce systematically biased outcomes for specific groups, particularly those defined by protected attributes such as race, gender, or age. Evaluating fairness is a critical aspect of ML model development, as biased models can perpetuate structural inequalities. The {fairmetrics} R package offers a user-friendly framework for rigorously evaluating numerous group-based fairness criteria, including metrics based on independence (e.g., statistical parity), separation (e.g., equalized odds), and sufficiency (e.g., predictive parity). Group-based fairness criteria assess whether a model is equally accurate or well-calibrated across a set of predefined groups so that appropriate bias mitigation strategies can be implemented. {fairmetrics} provides both point and interval estimates for multiple metrics through a convenient wrapper function and includes an example dataset derived from the Medical Information Mart for Intensive Care, version II (MIMIC-II) database (Goldberger et al., 2000; Raffa, 2016).