# MaDI-Bench: An End-to-End Data Integration Benchmark

arXiv: https://arxiv.org/abs/2606.30371 · HTML: https://mlsanity.com/paper/2606.30371/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 15 of 15)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [Valentine: Evaluating Matching Techniques for Dataset Discovery](https://mlsanity.com/paper/2010.07386/) (2020)
- [Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.16884/) (2024)
- [Entity Matching using Large Language Models](https://mlsanity.com/paper/2310.11244/) (2023)
- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [Large Language Model-based Data Science Agent: A Survey](https://mlsanity.com/paper/2508.02744/) (2025)
- [WDC Products: A Multi-Dimensional Entity Matching Benchmark](https://mlsanity.com/paper/2301.09521/) (2023)
- [Magneto: Combining Small and Large Language Models for Schema Matching](https://mlsanity.com/paper/2412.08194/) (2024)
- [ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines](https://mlsanity.com/paper/2504.04808/) (2025)
- [Alaska: A Flexible Benchmark for Data Integration Tasks](https://mlsanity.com/paper/2101.11259/) (2021)
- [SC-Block: Supervised Contrastive Blocking within Entity Resolution Pipelines](https://mlsanity.com/paper/2303.03132/) (2023)
- [Automatic End-to-End Data Integration using Large Language Models](https://mlsanity.com/paper/2603.10547/) (2026)
- [Evaluation of Pipelines for Data Integration into Knowledge Graphs](https://mlsanity.com/paper/2605.22304/) (2026)

## Built on by (most cited first, 1 of 1)

- [Corporate-Family Resolution Is Not a String-Matching Problem: A Public Benchmark Stratified by Name Visibility](https://mlsanity.com/paper/2609.04269/) (2026)

## Abstract (arXiv)

Data integration is the process of combining data from multiple, heterogeneous sources into a consistent, unified representation. Data integration involves a sequence of interdependent tasks including schema matching, value normalization, blocking, entity matching, and data fusion. Existing table-based benchmarks either evaluate these steps in isolation or cover only incomplete versions of the data integration pipeline, omitting specific steps. The lack of public end-to-end data integration benchmarks hinders research on data integration methods that address the integration process as a whole and account for the interdependencies among the different tasks. This paper fills this gap by introducing the Mannheim Data Integration Benchmark (MaDI-Bench), the first benchmark for the end-to-end integration of relational tables covering all steps of the integration process. MaDI-Bench contributes (i) a set of end-to-end data integration tasks spanning several application domains, each requiring the full schema matching, value normalization, entity matching, and data fusion pipeline, and (ii) a generic method for deriving task variants that mitigates rapid benchmark saturation as data integration systems advance. We validate the benchmark using human-engineered pipelines, a best-of-breed pipeline, an LLM workflow, and a pipeline written by a coding agent. The validation demonstrates the utility of the benchmark for measuring the step-wise as well as the end-to-end performance of data integration pipelines. All benchmark artifacts are available for public download.