# Valentine: Evaluating Matching Techniques for Dataset Discovery

arXiv: https://arxiv.org/abs/2010.07386 · HTML: https://mlsanity.com/paper/2010.07386/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 30 of 35)

- [Table-GPT: Table-tuned GPT for Diverse Table Tasks](https://mlsanity.com/paper/2310.09263/) (2023)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [Jellyfish: A Large Language Model for Data Preprocessing](https://mlsanity.com/paper/2312.01678/) (2023)
- [Schema Matching with Large Language Models: an Experimental Study](https://mlsanity.com/paper/2407.11852/) (2024)
- [Magneto: Combining Small and Large Language Models for Schema Matching](https://mlsanity.com/paper/2412.08194/) (2024)
- [DeepJoin: Joinable Table Discovery with Pre-trained Language Models](https://mlsanity.com/paper/2212.07588/) (2022)
- [MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.05587/) (2025)
- [NameGuess: Column Name Expansion for Tabular Data](https://mlsanity.com/paper/2310.13196/) (2023)
- [LakeBench: Benchmarks for Data Discovery over Data Lakes](https://mlsanity.com/paper/2307.04217/) (2023)
- [Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Generator-Validator Fine-tuning](https://mlsanity.com/paper/2410.12164/) (2024)
- [Unsupervised Matching of Data and Text](https://mlsanity.com/paper/2112.08776/) (2021)
- [AutoG: Towards automatic graph construction from tabular data](https://mlsanity.com/paper/2501.15282/) (2025)
- [Empowering Tabular Data Preparation with Language Models: Why and How?](https://mlsanity.com/paper/2508.01556/) (2025)
- [Interactive Data Harmonization with LLM Agents: Opportunities and Challenges](https://mlsanity.com/paper/2502.07132/) (2025)
- [Sheet As Token: A Graph-Enhanced Representation for Multi-Sheet Spreadsheet Understanding](https://mlsanity.com/paper/2605.05811/) (2026)
- [Generative Benchmark Creation for Table Union Search](https://mlsanity.com/paper/2308.03883/) (2023)
- [Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods](https://mlsanity.com/paper/2403.08352/) (2024)
- [TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes](https://mlsanity.com/paper/2407.01619/) (2024)
- [Prompt-Matcher: Leveraging Large Models to Reduce Uncertainty in Schema Matching Results](https://mlsanity.com/paper/2408.14507/) (2024)
- [Evaluation of Pipelines for Data Integration into Knowledge Graphs](https://mlsanity.com/paper/2605.22304/) (2026)
- [RACT: Retrieval Augmented Column-Table Learning and Prediction for Multi-Table Schema Matching](https://mlsanity.com/paper/2606.07843/) (2026)
- [MaDI-Bench: An End-to-End Data Integration Benchmark](https://mlsanity.com/paper/2606.30371/) (2026)
- [SINT-Flow: Schema Integration using Large Language Model Workflows](https://mlsanity.com/paper/2607.24492/) (2026)
- [Deep Learning to Jointly Schema Match, Impute, and Transform Databases](https://mlsanity.com/paper/2207.03536/) (2022)
- [Conceptual Schema Inference for Tabular Datasets using Large Language Models](https://mlsanity.com/paper/2509.04632/) (2025)
- [CONE: Embeddings for Complex Numerical Data Preserving Unit and Variable Semantics](https://mlsanity.com/paper/2603.04741/) (2026)
- [DBAutoDoc: Automated Discovery and Documentation of Undocumented Database Schemas via Statistical Analysis and Iterative LLM Refinement](https://mlsanity.com/paper/2603.23050/) (2026)
- [BDI-Kit Demo: A Toolkit for Programmable and Conversational Data Harmonization](https://mlsanity.com/paper/2604.06405/) (2026)
- [SemStruct: Contextualizing Semantic Embeddings with Structural Information for Schema Matching](https://mlsanity.com/paper/2605.30729/) (2026)
- [When Helping Hurts and How to Fix It: Multi-Agent Debate for Data Cleaning](https://mlsanity.com/paper/2606.02866/) (2026)

## Abstract (arXiv)

Data scientists today search large data lakes to discover and integrate datasets. In order to bring together disparate data sources, dataset discovery methods rely on some form of schema matching: the process of establishing correspondences between datasets. Traditionally, schema matching has been used to find matching pairs of columns between a source and a target schema. However, the use of schema matching in dataset discovery methods differs from its original use. Nowadays schema matching serves as a building block for indicating and ranking inter-dataset relationships. Surprisingly, although a discovery method's success relies highly on the quality of the underlying matching algorithms, the latest discovery methods employ existing schema matching algorithms in an ad-hoc fashion due to the lack of openly-available datasets with ground truth, reference method implementations, and evaluation metrics. In this paper, we aim to rectify the problem of evaluating the effectiveness and efficiency of schema matching methods for the specific needs of dataset discovery. To this end, we propose Valentine, an extensible open-source experiment suite to execute and organize large-scale automated matching experiments on tabular data. Valentine includes implementations of seminal schema matching methods that we either implemented from scratch (due to absence of open source code) or imported from open repositories. The contributions of Valentine are: i) the definition of four schema matching scenarios as encountered in dataset discovery methods, ii) a principled dataset fabrication process tailored to the scope of dataset discovery methods and iii) the most comprehensive evaluation of schema matching techniques to date, offering insight on the strengths and weaknesses of existing techniques, that can serve as a guide for employing schema matching in future dataset discovery methods.