# Efficient Joinable Table Discovery in Data Lakes: A High-Dimensional Similarity-Based Approach

arXiv: https://arxiv.org/abs/2010.13273 · HTML: https://mlsanity.com/paper/2010.13273/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2020.

## Built on by (most cited first, 23 of 23)

- [TransTab: Learning Transferable Tabular Transformers Across Tables](https://mlsanity.com/paper/2205.09328/) (2022)
- [TabLib: A Dataset of 627M Tables with Context](https://mlsanity.com/paper/2310.07875/) (2023)
- [DeepJoin: Joinable Table Discovery with Pre-trained Language Models](https://mlsanity.com/paper/2212.07588/) (2022)
- [Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AI](https://mlsanity.com/paper/2407.21523/) (2024)
- [4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs](https://mlsanity.com/paper/2404.18209/) (2024)
- [DBCopilot: Natural Language Querying over Massive Databases via Schema Routing](https://mlsanity.com/paper/2312.03463/) (2023)
- [Cross Modal Data Discovery over Structured and Unstructured Data Lakes](https://mlsanity.com/paper/2306.00932/) (2023)
- [Snoopy: Effective and Efficient Semantic Join Discovery via Proxy Columns](https://mlsanity.com/paper/2502.16813/) (2025)
- [DALL-M: Context-Aware Clinical Data Augmentation with LLMs](https://mlsanity.com/paper/2407.08227/) (2024)
- [Taxonomy Inference for Tabular Data Using Large Language Models](https://mlsanity.com/paper/2503.21810/) (2025)
- [HyperJoin: LLM-augmented Hypergraph Link Prediction for Joinable Table Discovery](https://mlsanity.com/paper/2601.01015/) (2026)
- [TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes](https://mlsanity.com/paper/2407.01619/) (2024)
- [Leveraging Foundation Language Models (FLMs) for Automated Cohort Extraction from Large EHR Databases](https://mlsanity.com/paper/2412.11472/) (2024)
- [A Survey on Semantics in Automated Data Science](https://mlsanity.com/paper/2205.08018/) (2022)
- [A Vision for Semantically Enriched Data Science](https://mlsanity.com/paper/2303.01378/) (2023)
- [Xling: A Learned Filter Framework for Accelerating High-Dimensional Approximate Similarity Join](https://mlsanity.com/paper/2402.13397/) (2024)
- [Relationships are Complicated! An Analysis of Relationships Between Datasets on the Web](https://mlsanity.com/paper/2408.14636/) (2024)
- [Conceptual Schema Inference for Tabular Datasets using Large Language Models](https://mlsanity.com/paper/2509.04632/) (2025)
- [Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Data](https://mlsanity.com/paper/2602.02025/) (2026)
- [Nexus: Inferring Join Graphs from Metadata Alone via Iterative Low-Rank Matrix Completion](https://mlsanity.com/paper/2602.08186/) (2026)
- [Alignment-Guided Largest Table Overlap Size Estimation](https://mlsanity.com/paper/2607.03049/) (2026)
- [MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery](https://mlsanity.com/paper/2607.21781/) (2026)
- [TabJoinBench: A Benchmark for Joinable Table Discovery](https://mlsanity.com/paper/2610.00817/) (2026)

## Abstract (arXiv)

Finding joinable tables in data lakes is key procedure in many applications such as data integration, data augmentation, data analysis, and data market. Traditional approaches that find equi-joinable tables are unable to deal with misspellings and different formats, nor do they capture any semantic joins. In this paper, we propose PEXESO, a framework for joinable table discovery in data lakes. We embed textual values as high-dimensional vectors and join columns under similarity predicates on high-dimensional vectors, hence to address the limitations of equi-join approaches and identify more meaningful results. To efficiently find joinable tables with similarity, we propose a block-and-verify method that utilizes pivot-based filtering. A partitioning technique is developed to cope with the case when the data lake is large and the index cannot fit in main memory. An experimental evaluation on real datasets shows that our solution identifies substantially more tables than equi-joins and outperforms other similarity-based options, and the join results are useful in data enrichment for machine learning tasks. The experiments also demonstrate the efficiency of the proposed method.