# Reducing Biases in Record Matching Through Scores Calibration

arXiv: https://arxiv.org/abs/2411.01685 · HTML: https://mlsanity.com/paper/2411.01685/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 3 Nov 2024.

## Builds on (most cited first, 28 of 28)

- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [FA*IR: A Fair Top-k Ranking Algorithm](https://mlsanity.com/paper/1706.06368/) (2017)
- [Fair Regression with Wasserstein Barycenters](https://mlsanity.com/paper/2006.07286/) (2020)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [Policy Learning for Fairness in Ranking](https://mlsanity.com/paper/1902.04056/) (2019)
- [Fairness in Rankings and Recommendations: An Overview](https://mlsanity.com/paper/2104.05994/) (2021)
- [DeepER -- Deep Entity Resolution](https://mlsanity.com/paper/1710.00597/) (2017)
- [The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric](https://mlsanity.com/paper/1902.05826/) (2019)
- [Low-resource Deep Entity Resolution with Transfer and Active Learning](https://mlsanity.com/paper/1906.08042/) (2019)
- [Interventions for Ranking in the Presence of Implicit Bias](https://mlsanity.com/paper/2001.08767/) (2020)
- [Balanced Ranking with Diversity Constraints](https://mlsanity.com/paper/1906.01747/) (2019)
- [A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching](https://mlsanity.com/paper/2003.13114/) (2020)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)
- [Minimax AUC Fairness: Efficient Algorithm with Provable Convergence](https://mlsanity.com/paper/2208.10451/) (2022)
- [Wasserstein-based fairness interpretability framework for machine learning models](https://mlsanity.com/paper/2011.03156/) (2020)
- [A General Framework for Fair Regression](https://mlsanity.com/paper/1810.05041/) (2018)
- [The Misuse of AUC: What High Impact Risk Assessment Gets Wrong](https://mlsanity.com/paper/2305.18159/) (2023)
- [Through the Fairness Lens: Experimental Analysis and Evaluation of Entity Matching](https://mlsanity.com/paper/2307.02726/) (2023)
- [Repairing Regressors for Fair Binary Classification at Any Decision Threshold](https://mlsanity.com/paper/2203.07490/) (2022)
- [Demographic Parity Constrained Minimax Optimal Regression under Linear Model](https://mlsanity.com/paper/2206.11546/) (2022)
- [Fair Rank Aggregation](https://mlsanity.com/paper/2308.10499/) (2023)
- [Threshold-Independent Fair Matching through Score Calibration](https://mlsanity.com/paper/2405.20051/) (2024)
- [FairEM360: A Suite for Responsible Entity Matching](https://mlsanity.com/paper/2404.07354/) (2024)

## Abstract (arXiv)

Record matching models typically output a real-valued matching score that is later consumed through thresholding, ranking, or human review. While fairness in record matching has mostly been assessed using binary decisions at a fixed threshold, such evaluations can miss systematic disparities in the entire score distribution and can yield conclusions that change with the chosen threshold. We introduce a threshold-independent notion of score bias that extends standard group-fairness criteria-demographic parity (DP), equal opportunity (EO), and equalized odds (EOD)-from binary outputs to score functions by integrating group-wise metric gaps over all thresholds. Using this metric, we empirically show that several state-of-the-art deep matchers can exhibit substantial score bias even when appearing fair at commonly used thresholds. To mitigate these disparities without retraining the underlying matcher, we propose two model-agnostic post-processing methods that only require score evaluations on an (unlabeled) calibration set. Calib targets DP by aligning minority/majority score distributions to a common Wasserstein barycenter via a quantile-based optimal-transport map, with finite-sample guarantees on both residual DP bias and score distortion. C-Calib extends this idea to label-dependent notions (EO/EOD) by performing barycenter alignment conditionally on an estimated label, and we characterize how its guarantees depend on both sample size and label-estimation error. Experiments on standard record-matching benchmarks and multiple neural matchers confirm that Calib and C-Calib substantially reduce score bias with minimal loss in accuracy.