# WarpGate: A Semantic Join Discovery System for Cloud Data Warehouses

arXiv: https://arxiv.org/abs/2212.14155 · HTML: https://mlsanity.com/paper/2212.14155/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2022.

## Built on by (most cited first, 13 of 13)

- [Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval](https://mlsanity.com/paper/2404.09889/) (2024)
- [CHORUS: Foundation Models for Unified Data Discovery and Exploration](https://mlsanity.com/paper/2306.09610/) (2023)
- [TabLib: A Dataset of 627M Tables with Context](https://mlsanity.com/paper/2310.07875/) (2023)
- [Observatory: Characterizing Embeddings of Relational Tables](https://mlsanity.com/paper/2310.07736/) (2023)
- [UniDM: A Unified Framework for Data Manipulation with Large Language Models](https://mlsanity.com/paper/2405.06510/) (2024)
- [Retrieve, Merge, Predict: Augmenting Tables with Data Lakes](https://mlsanity.com/paper/2402.06282/) (2024)
- [Snoopy: Effective and Efficient Semantic Join Discovery via Proxy Columns](https://mlsanity.com/paper/2502.16813/) (2025)
- [TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes](https://mlsanity.com/paper/2407.01619/) (2024)
- [CORE-T: COherent REtrieval of Tables for Text-to-SQL](https://mlsanity.com/paper/2601.13111/) (2026)
- [Nexus: Inferring Join Graphs from Metadata Alone via Iterative Low-Rank Matrix Completion](https://mlsanity.com/paper/2602.08186/) (2026)
- [SAGE: Structure Aware Graph Expansion for Retrieval of Heterogeneous Data](https://mlsanity.com/paper/2602.16964/) (2026)
- [MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery](https://mlsanity.com/paper/2607.21781/) (2026)
- [TabJoinBench: A Benchmark for Joinable Table Discovery](https://mlsanity.com/paper/2610.00817/) (2026)

## Abstract (arXiv)

Data discovery is a major challenge in enterprise data analysis: users often struggle to find data relevant to their analysis goals or even to navigate through data across data sources, each of which may easily contain thousands of tables. One common user need is to discover tables joinable with a given table. This need is particularly critical because join is a ubiquitous operation in data analysis, and join paths are mostly obscure to users, especially across databases. Furthermore, users are typically interested in finding ``semantically'' joinable tables: with columns that can be transformed to become joinable even if they are not joinable as currently represented in the data store. We present WarpGate, a system prototype for data discovery over cloud data warehouses. WarpGate implements an embedding-based solution to semantic join discovery, which encodes columns into high-dimensional vector space such that joinable columns map to points that are near each other. Through experiments on several table corpora, we show that WarpGate (i) captures semantic relationships between tables, especially those across databases, and (ii) is sample efficient and thus scalable to very large tables of millions of rows. We also showcase an application of WarpGate within an enterprise product for cloud data analytics.