# WDC Products: A Multi-Dimensional Entity Matching Benchmark arXiv: https://arxiv.org/abs/2301.09521 ยท HTML: https://mlsanity.com/paper/2301.09521/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Builds on 21 of its 55 references that are in the map. - First posted on arXiv 23 Jan 2023. ## Builds on (most cited first, 21 of 21) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015) - [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020) - [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021) - [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021) - [Optimizing Dense Retrieval Model Training with Hard Negatives](https://mlsanity.com/paper/2104.08051/) (2021) - [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020) - [In Search of Robust Measures of Generalization](https://mlsanity.com/paper/2010.11924/) (2020) - [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021) - [Neural Networks for Entity Matching: A Survey](https://mlsanity.com/paper/2010.11075/) (2020) - [Supervised Contrastive Learning for Product Matching](https://mlsanity.com/paper/2202.02098/) (2022) - [Sudowoodo: Contrastive Self-supervised Learning for Multi-purpose Data Integration and Preparation](https://mlsanity.com/paper/2207.04122/) (2022) - [Machamp: A Generalized Entity Matching Benchmark](https://mlsanity.com/paper/2106.08455/) (2021) - [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022) - [Deep Hierarchical Classification for Category Prediction in E-commerce System](https://mlsanity.com/paper/2005.06692/) (2020) - [Alaska: A Flexible Benchmark for Data Integration Tasks](https://mlsanity.com/paper/2101.11259/) (2021) - [FlexER: Flexible Entity Resolution for Multiple Intents](https://mlsanity.com/paper/2209.07569/) (2022) - [CollaborER: A Self-supervised Entity Resolution Framework Using Multi-features Collaboration](https://mlsanity.com/paper/2108.08090/) (2021) ## Built on by (most cited first, 17 of 17) - [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023) - [Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.16884/) (2024) - [Entity Matching using Large Language Models](https://mlsanity.com/paper/2310.11244/) (2023) - [Using ChatGPT for Entity Matching](https://mlsanity.com/paper/2305.03423/) (2023) - [Fine-tuning Large Language Models for Entity Matching](https://mlsanity.com/paper/2409.08185/) (2024) - [Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs](https://mlsanity.com/paper/2601.17058/) (2026) - [SC-Block: Supervised Contrastive Blocking within Entity Resolution Pipelines](https://mlsanity.com/paper/2303.03132/) (2023) - [TransClean: Finding False Positives in Multi-Source Entity Matching under Real-World Conditions via Transitive Consistency](https://mlsanity.com/paper/2506.04006/) (2025) - [Entity Resolution via Batched Oracle Queries](https://mlsanity.com/paper/2606.24407/) (2026) - [Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching](https://mlsanity.com/paper/2606.27342/) (2026) - [MaDI-Bench: An End-to-End Data Integration Benchmark](https://mlsanity.com/paper/2606.30371/) (2026) - [APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching](https://mlsanity.com/paper/2405.04820/) (2024) - [GraLMatch: Matching Groups of Entities with Graphs and Language Models](https://mlsanity.com/paper/2406.15015/) (2024) - [OpenSanctions Pairs: Large-Scale Entity Matching with LLMs](https://mlsanity.com/paper/2603.11051/) (2026) - [Beyond Scale and Generation: Understanding Language Model-based Entity Matching](https://mlsanity.com/paper/2607.24688/) (2026) - [Corporate-Family Resolution Is Not a String-Matching Problem: A Public Benchmark Stratified by Name Visibility](https://mlsanity.com/paper/2609.04269/) (2026) - [Billiger.de Products: A Bilingual Entity Matching Benchmark](https://mlsanity.com/paper/2609.37713/) (2026) ## Abstract (arXiv) The difficulty of an entity matching task depends on a combination of multiple factors such as the amount of corner-case pairs, the fraction of entities in the test set that have not been seen during training, and the size of the development set. Current entity matching benchmarks usually represent single points in the space along such dimensions or they provide for the evaluation of matching methods along a single dimension, for instance the amount of training data. This paper presents WDC Products, an entity matching benchmark which provides for the systematic evaluation of matching systems along combinations of three dimensions while relying on real-world data. The three dimensions are (i) amount of corner-cases (ii) generalization to unseen entities, and (iii) development set size (training set plus validation set). Generalization to unseen entities is a dimension not covered by any of the existing English-language benchmarks yet but is crucial for evaluating the robustness of entity matching systems. Instead of learning how to match entity pairs, entity matching can also be formulated as a multi-class classification task that requires the matcher to recognize individual entities. WDC Products is the first benchmark that provides a pair-wise and a multi-class formulation of the same tasks. We evaluate WDC Products using several state-of-the-art matching systems, including Ditto, HierGAT, and R-SupCon. The evaluation shows that all matching systems struggle with unseen entities to varying degrees. It also shows that for entity matching contrastive learning is more training data efficient compared to cross-encoders.