# A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching

arXiv: https://arxiv.org/abs/2003.13114 · HTML: https://mlsanity.com/paper/2003.13114/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [DataPerf: Benchmarks for Data-Centric AI Development](https://mlsanity.com/paper/2207.10062/) (2022)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Deep Indexed Active Learning for Matching Heterogeneous Entity Representations](https://mlsanity.com/paper/2104.03986/) (2021)
- [Data Cleaning and Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2310.01765/) (2023)
- [FlexER: Flexible Entity Resolution for Multiple Intents](https://mlsanity.com/paper/2209.07569/) (2022)
- [The Battleship Approach to the Low Resource Entity Matching Problem](https://mlsanity.com/paper/2311.15685/) (2023)
- [Leveraging Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.20624/) (2024)
- [Finding Needles in the Haystack: Transductive Active Labeling in Ecology](https://mlsanity.com/paper/2606.03821/) (2026)
- [Physics-Coupled Spatio-Temporal Active Learning for Dynamical Systems](https://mlsanity.com/paper/2108.05385/) (2021)
- [HumanAL: Calibrating Human Matching Beyond a Single Task](https://mlsanity.com/paper/2205.03209/) (2022)
- [Disambiguation of Company names via Deep Recurrent Networks](https://mlsanity.com/paper/2303.05391/) (2023)
- [Does Deep Active Learning Work in the Wild?](https://mlsanity.com/paper/2302.00098/) (2023)
- [TransClean: Finding False Positives in Multi-Source Entity Matching under Real-World Conditions via Transitive Consistency](https://mlsanity.com/paper/2506.04006/) (2025)
- [Reducing Biases in Record Matching Through Scores Calibration](https://mlsanity.com/paper/2411.01685/) (2024)
- [Monotone Classification with Relative Approximations](https://mlsanity.com/paper/2506.10775/) (2025)
- [Adaptive Graph Refinement and Label Propagation with LLMs for Cost-Effective Entity Resolution](https://mlsanity.com/paper/2605.25814/) (2026)

## Abstract (arXiv)

Entity Matching (EM) is a core data cleaning task, aiming to identify different mentions of the same real-world entity. Active learning is one way to address the challenge of scarce labeled data in practice, by dynamically collecting the necessary examples to be labeled by an Oracle and refining the learned model (classifier) upon them. In this paper, we build a unified active learning benchmark framework for EM that allows users to easily combine different learning algorithms with applicable example selection algorithms. The goal of the framework is to enable concrete guidelines for practitioners as to what active learning combinations will work well for EM. Towards this, we perform comprehensive experiments on publicly available EM datasets from product and publication domains to evaluate active learning methods, using a variety of metrics including EM quality, #labels and example selection latencies. Our most surprising result finds that active learning with fewer labels can learn a classifier of comparable quality as supervised learning. In fact, for several of the datasets, we show that there is an active learning combination that beats the state-of-the-art supervised learning result. Our framework also includes novel optimizations that improve the quality of the learned model by roughly 9% in terms of F1-score and reduce example selection latencies by up to 10x without affecting the quality of the model.