# Deep Indexed Active Learning for Matching Heterogeneous Entity Representations

arXiv: https://arxiv.org/abs/2104.03986 · HTML: https://mlsanity.com/paper/2104.03986/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 73 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [DeepER -- Deep Entity Resolution](https://mlsanity.com/paper/1710.00597/) (2017)
- [Low-resource Deep Entity Resolution with Transfer and Active Learning](https://mlsanity.com/paper/1906.08042/) (2019)
- [Neural Networks for Entity Matching: A Survey](https://mlsanity.com/paper/2010.11075/) (2020)
- [A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching](https://mlsanity.com/paper/2003.13114/) (2020)
- [AutoBlock: A Hands-off Blocking Framework for Entity Matching](https://mlsanity.com/paper/1912.03417/) (2019)
- [On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks](https://mlsanity.com/paper/2010.05103/) (2020)
- [A High-Quality Multilingual Dataset for Structured Documentation Translation](https://mlsanity.com/paper/2006.13425/) (2020)
- [CorDEL: A Contrastive Deep Learning Approach for Entity Linkage](https://mlsanity.com/paper/2009.07203/) (2020)
- [MinoanER: Schema-Agnostic, Non-Iterative, Massively Parallel Resolution of Web Entities](https://mlsanity.com/paper/1905.06170/) (2019)
- [Efficient and Effective ER with Progressive Blocking](https://mlsanity.com/paper/2005.14326/) (2020)

## Built on by (most cited first, 9 of 9)

- [Deep Active Alignment of Knowledge Graph Entities and Schemata](https://mlsanity.com/paper/2304.04389/) (2023)
- [A Critical Re-evaluation of Benchmark Datasets for (Deep) Learning-Based Matching Algorithms](https://mlsanity.com/paper/2307.01231/) (2023)
- [The Battleship Approach to the Low Resource Entity Matching Problem](https://mlsanity.com/paper/2311.15685/) (2023)
- [Learning variant product relationship and variation attributes from e-commerce website structures](https://mlsanity.com/paper/2410.02779/) (2024)
- [HumanAL: Calibrating Human Matching Beyond a Single Task](https://mlsanity.com/paper/2205.03209/) (2022)
- [A Pre-trained Data Deduplication Model based on Active Learning](https://mlsanity.com/paper/2308.00721/) (2023)
- [Towards Universal Dense Blocking for Entity Resolution](https://mlsanity.com/paper/2404.14831/) (2024)
- [3dSAGER: Geospatial Entity Resolution over 3D Objects (Technical Report)](https://mlsanity.com/paper/2511.06300/) (2025)
- [Labeling Training Data for Entity Matching Using Large Language Models](https://mlsanity.com/paper/2606.28823/) (2026)

## Abstract (arXiv)

Given two large lists of records, the task in entity resolution (ER) is to find the pairs from the Cartesian product of the lists that correspond to the same real world entity. Typically, passive learning methods on such tasks require large amounts of labeled data to yield useful models. Active Learning is a promising approach for ER in low resource settings. However, the search space, to find informative samples for the user to label, grows quadratically for instance-pair tasks making active learning hard to scale. Previous works, in this setting, rely on hand-crafted predicates, pre-trained language model embeddings, or rule learning to prune away unlikely pairs from the Cartesian product. This blocking step can miss out on important regions in the product space leading to low recall. We propose DIAL, a scalable active learning approach that jointly learns embeddings to maximize recall for blocking and accuracy for matching blocked pairs. DIAL uses an Index-By-Committee framework, where each committee member learns representations based on powerful pre-trained transformer language models. We highlight surprising differences between the matcher and the blocker in the creation of the training data and the objective used to train their parameters. Experiments on five benchmark datasets and a multilingual record matching dataset show the effectiveness of our approach in terms of precision, recall and running time. Code is available at https://github.com/ArjitJ/DIAL