# Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning

arXiv: https://arxiv.org/abs/2210.01922 · HTML: https://mlsanity.com/paper/2210.01922/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2022.

## Built on by (most cited first, 30 of 38)

- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [Jellyfish: A Large Language Model for Data Preprocessing](https://mlsanity.com/paper/2312.01678/) (2023)
- [Magneto: Combining Small and Large Language Models for Schema Matching](https://mlsanity.com/paper/2412.08194/) (2024)
- [Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AI](https://mlsanity.com/paper/2407.21523/) (2024)
- [Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks](https://mlsanity.com/paper/2310.00789/) (2023)
- [DBCopilot: Natural Language Querying over Massive Databases via Schema Routing](https://mlsanity.com/paper/2312.03463/) (2023)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [Retrieve, Merge, Predict: Augmenting Tables with Data Lakes](https://mlsanity.com/paper/2402.06282/) (2024)
- [Snoopy: Effective and Efficient Semantic Join Discovery via Proxy Columns](https://mlsanity.com/paper/2502.16813/) (2025)
- [Empowering Tabular Data Preparation with Language Models: Why and How?](https://mlsanity.com/paper/2508.01556/) (2025)
- [Table Integration in Data Lakes Unleashed: Pairwise Integrability Judgment, Integrable Set Discovery, and Multi-Tuple Conflict Resolution](https://mlsanity.com/paper/2412.00324/) (2024)
- [KGLiDS: A Platform for Semantic Abstraction, Linking, and Automation of Data Science](https://mlsanity.com/paper/2303.02204/) (2023)
- [Model Lakes](https://mlsanity.com/paper/2403.02327/) (2024)
- [CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems](https://mlsanity.com/paper/2406.00583/) (2024)
- [DALL-M: Context-Aware Clinical Data Augmentation with LLMs](https://mlsanity.com/paper/2407.08227/) (2024)
- [Taxonomy Inference for Tabular Data Using Large Language Models](https://mlsanity.com/paper/2503.21810/) (2025)
- [HyperJoin: LLM-augmented Hypergraph Link Prediction for Joinable Table Discovery](https://mlsanity.com/paper/2601.01015/) (2026)
- [Generative Benchmark Creation for Table Union Search](https://mlsanity.com/paper/2308.03883/) (2023)
- [Towards Universal Dense Blocking for Entity Resolution](https://mlsanity.com/paper/2404.14831/) (2024)
- [TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes](https://mlsanity.com/paper/2407.01619/) (2024)
- [Leveraging Foundation Language Models (FLMs) for Automated Cohort Extraction from Large EHR Databases](https://mlsanity.com/paper/2412.11472/) (2024)
- [Generating Skyline Datasets for Data Science Models](https://mlsanity.com/paper/2502.11262/) (2025)
- [Something's Fishy In The Data Lake: A Critical Re-evaluation of Table Union Search Benchmarks](https://mlsanity.com/paper/2505.21329/) (2025)
- [Humans, Machine Learning, and Language Models in Union: A Cognitive Study on Table Unionability](https://mlsanity.com/paper/2506.12990/) (2025)
- [ML-Asset Management: Curation, Discovery, and Utilization](https://mlsanity.com/paper/2509.23577/) (2025)
- [LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake](https://mlsanity.com/paper/2606.10460/) (2026)
- [WikiDBGraph: A Data Management Benchmark Suite for Collaborative Learning over Database Silos](https://mlsanity.com/paper/2505.16635/) (2025)
- [Conceptual Schema Inference for Tabular Datasets using Large Language Models](https://mlsanity.com/paper/2509.04632/) (2025)
- [SAVeD: Semantic Aware Version Discovery](https://mlsanity.com/paper/2511.17298/) (2025)
- [ModelTables: A Corpus of Tables about Models](https://mlsanity.com/paper/2512.16106/) (2025)

## Abstract (arXiv)

Dataset discovery from data lakes is essential in many real application scenarios. In this paper, we propose Starmie, an end-to-end framework for dataset discovery from data lakes (with table union search as the main use case). Our proposed framework features a contrastive learning method to train column encoders from pre-trained language models in a fully unsupervised manner. The column encoder of Starmie captures the rich contextual semantic information within tables by leveraging a contrastive multi-column pre-training strategy. We utilize the cosine similarity between column embedding vectors as the column unionability score and propose a filter-and-verification framework that allows exploring a variety of design choices to compute the unionability score between two tables accordingly. Empirical evaluation results on real table benchmark datasets show that Starmie outperforms the best-known solutions in the effectiveness of table union search by 6.8 in MAP and recall. Moreover, Starmie is the first to employ the HNSW (Hierarchical Navigable Small World) index for accelerate query processing of table union search which provides a 3,000X performance gain over the linear scan baseline and a 400X performance gain over an LSH index (the state-of-the-art solution for data lake indexing).