# Cross Modal Data Discovery over Structured and Unstructured Data Lakes

arXiv: https://arxiv.org/abs/2306.00932 · HTML: https://mlsanity.com/paper/2306.00932/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 76 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [Sherlock: A Deep Learning Approach to Semantic Data Type Detection](https://mlsanity.com/paper/1905.10688/) (2019)
- [Approximate Nearest Neighbor Search on High Dimensional Data --- Experiments, Analyses, and Improvement (v1.0)](https://mlsanity.com/paper/1610.02455/) (2016)
- [Dataset Discovery in Data Lakes](https://mlsanity.com/paper/2011.10427/) (2020)
- [LSH Ensemble: Internet-Scale Domain Search](https://mlsanity.com/paper/1603.07410/) (2016)
- [Efficient Joinable Table Discovery in Data Lakes: A High-Dimensional Similarity-Based Approach](https://mlsanity.com/paper/2010.13273/) (2020)
- [Goals, Process, and Challenges of Exploratory Data Analysis: An Interview Study](https://mlsanity.com/paper/1911.00568/) (2019)
- [Siamese CBOW: Optimizing Word Embeddings for Sentence Representations](https://mlsanity.com/paper/1606.04640/) (2016)
- [DomainNet: Homograph Detection for Data Lake Disambiguation](https://mlsanity.com/paper/2103.09940/) (2021)
- [Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins](https://mlsanity.com/paper/2106.01501/) (2021)
- [Unsupervised Matching of Data and Text](https://mlsanity.com/paper/2112.08776/) (2021)
- [Cognitive Database: A Step towards Endowing Relational Databases with Artificial Intelligence Capabilities](https://mlsanity.com/paper/1712.07199/) (2017)
- [Termite: A System for Tunneling Through Heterogeneous Data](https://mlsanity.com/paper/1903.05008/) (2019)
- [Graph integration of structured, semistructured and unstructured data for data journalism](https://mlsanity.com/paper/2007.12488/) (2020)
- [The General Pair-based Weighting Loss for Deep Metric Learning](https://mlsanity.com/paper/1905.12837/) (2019)
- [An IDEA: An Ingestion Framework for Data Enrichment in AsterixDB](https://mlsanity.com/paper/1902.08271/) (2019)

## Built on by (most cited first, 5 of 5)

- [Snoopy: Effective and Efficient Semantic Join Discovery via Proxy Columns](https://mlsanity.com/paper/2502.16813/) (2025)
- [CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems](https://mlsanity.com/paper/2406.00583/) (2024)
- [Tabular Embeddings for Tables with Bi-Dimensional Hierarchical Metadata and Nesting](https://mlsanity.com/paper/2502.15819/) (2025)
- [LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes](https://mlsanity.com/paper/2607.12310/) (2026)
- [Discovery-Driven Integration of Disjoint Tables via Text](https://mlsanity.com/paper/2609.26658/) (2026)

## Abstract (arXiv)

Organizations are collecting increasingly large amounts of data for data driven decision making. These data are often dumped into a centralized repository, e.g., a data lake, consisting of thousands of structured and unstructured datasets. Perversely, such mixture of datasets makes the problem of discovering elements (e.g., tables or documents) that are relevant to a user's query or an analytical task very challenging. Despite the recent efforts in data discovery, the problem remains widely open especially in the two fronts of (1) discovering relationships and relatedness across structured and unstructured datasets where existing techniques suffer from either scalability, being customized for a specific problem type (e.g., entity matching or data integration), or demolishing the structural properties on its way, and (2) developing a holistic system for integrating various similarity measurements and sketches in an effective way to boost the discovery accuracy. In this paper, we propose a new data discovery system, named CMDL, for addressing these two limitations. CMDL supports the data discovery process over both structured and unstructured data while retaining the structural properties of tables.