# DeepER -- Deep Entity Resolution

arXiv: https://arxiv.org/abs/1710.00597 · HTML: https://mlsanity.com/paper/1710.00597/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2017.

## Built on by (most cited first, 30 of 54)

- [Entity Matching using Large Language Models](https://mlsanity.com/paper/2310.11244/) (2023)
- [Knowing the No-match: Entity Alignment with Dangling Cases](https://mlsanity.com/paper/2106.02248/) (2021)
- [VerifAI: Verified Generative AI](https://mlsanity.com/paper/2307.02796/) (2023)
- [Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration](https://mlsanity.com/paper/2312.03987/) (2023)
- [Pre-trained Embeddings for Entity Resolution: An Experimental Analysis [Experiment, Analysis & Benchmark]](https://mlsanity.com/paper/2304.12329/) (2023)
- [Experimental Analysis of Large-scale Learnable Vector Storage Compression](https://mlsanity.com/paper/2311.15578/) (2023)
- [Deep Indexed Active Learning for Matching Heterogeneous Entity Representations](https://mlsanity.com/paper/2104.03986/) (2021)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [Data Cleaning and Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2310.01765/) (2023)
- [Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs](https://mlsanity.com/paper/2601.17058/) (2026)
- [Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins](https://mlsanity.com/paper/2106.01501/) (2021)
- [Deep Transfer Learning for Multi-source Entity Linkage via Domain Adaptation](https://mlsanity.com/paper/2110.14509/) (2021)
- [Unsupervised Matching of Data and Text](https://mlsanity.com/paper/2112.08776/) (2021)
- [Deep Active Alignment of Knowledge Graph Entities and Schemata](https://mlsanity.com/paper/2304.04389/) (2023)
- [CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models](https://mlsanity.com/paper/2312.03256/) (2023)
- [UniDM: A Unified Framework for Data Manipulation with Large Language Models](https://mlsanity.com/paper/2405.06510/) (2024)
- [Demonstration of Panda: A Weakly Supervised Entity Matching System](https://mlsanity.com/paper/2106.10821/) (2021)
- [FlexER: Flexible Entity Resolution for Multiple Intents](https://mlsanity.com/paper/2209.07569/) (2022)
- [SC-Block: Supervised Contrastive Blocking within Entity Resolution Pipelines](https://mlsanity.com/paper/2303.03132/) (2023)
- [Learning from Natural Language Explanations for Generalizable Entity Matching](https://mlsanity.com/paper/2406.09330/) (2024)
- [Ground Truth Inference for Weakly Supervised Entity Matching](https://mlsanity.com/paper/2211.06975/) (2022)
- [PoWareMatch: a Quality-aware Deep Learning Approach to Improve Human Schema Matching](https://mlsanity.com/paper/2109.07321/) (2021)
- [It's AI Match: A Two-Step Approach for Schema Matching Using Embeddings](https://mlsanity.com/paper/2203.04366/) (2022)
- [A Critical Re-evaluation of Benchmark Datasets for (Deep) Learning-Based Matching Algorithms](https://mlsanity.com/paper/2307.01231/) (2023)
- [The Battleship Approach to the Low Resource Entity Matching Problem](https://mlsanity.com/paper/2311.15685/) (2023)
- [Whois? Deep Author Name Disambiguation using Bibliographic Data](https://mlsanity.com/paper/2207.04772/) (2022)
- [Evaluating Blocking Biases in Entity Matching](https://mlsanity.com/paper/2409.16410/) (2024)
- [Table Integration in Data Lakes Unleashed: Pairwise Integrability Judgment, Integrable Set Discovery, and Multi-Tuple Conflict Resolution](https://mlsanity.com/paper/2412.00324/) (2024)
- [Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making](https://mlsanity.com/paper/2106.04174/) (2021)
- [LEMON: Explainable Entity Matching](https://mlsanity.com/paper/2110.00516/) (2021)

## Abstract (arXiv)

Entity resolution (ER) is a key data integration problem. Despite the efforts in 70+ years in all aspects of ER, there is still a high demand for democratizing ER - humans are heavily involved in labeling data, performing feature engineering, tuning parameters, and defining blocking functions. With the recent advances in deep learning, in particular distributed representation of words (a.k.a. word embeddings), we present a novel ER system, called DeepER, that achieves good accuracy, high efficiency, as well as ease-of-use (i.e., much less human efforts). For accuracy, we use sophisticated composition methods, namely uni- and bi-directional recurrent neural networks (RNNs) with long short term memory (LSTM) hidden units, to convert each tuple to a distributed representation (i.e., a vector), which can in turn be used to effectively capture similarities between tuples. We consider both the case where pre-trained word embeddings are available as well the case where they are not; we present ways to learn and tune the distributed representations. For efficiency, we propose a locality sensitive hashing (LSH) based blocking approach that uses distributed representations of tuples; it takes all attributes of a tuple into consideration and produces much smaller blocks, compared with traditional methods that consider only a few attributes. For ease-of-use, DeepER requires much less human labeled data and does not need feature engineering, compared with traditional machine learning based approaches which require handcrafted features, and similarity functions along with their associated thresholds. We evaluate our algorithms on multiple datasets (including benchmarks, biomedical data, as well as multi-lingual data) and the extensive experimental results show that DeepER outperforms existing solutions.