# LEDD: Large Language Model-Empowered Data Discovery in Data Lakes

arXiv: https://arxiv.org/abs/2502.15182 · HTML: https://mlsanity.com/paper/2502.15182/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 13 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Builds on (most cited first, 1 of 1)

- [SANTOS: Relationship-based Semantic Table Union Search](https://mlsanity.com/paper/2209.13589/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs](https://mlsanity.com/paper/2601.17058/) (2026)
- [Cracking Vector Search Indexes](https://mlsanity.com/paper/2503.01823/) (2025)
- [Can AI autonomously build, operate, and use the entire data stack?](https://mlsanity.com/paper/2512.07926/) (2025)

## Abstract (arXiv)

Data discovery in data lakes with ever increasing datasets has long been recognized as a big challenge in the realm of data management, especially for semantic search of and hierarchical global catalog generation of tables. While large language models (LLMs) facilitate the processing of data semantics, challenges remain in architecting an end-to-end system that comprehensively exploits LLMs for the two semantics-related tasks. In this demo, we propose LEDD, an end-to-end system with an extensible architecture that leverages LLMs to provide hierarchical global catalogs with semantic meanings and semantic table search for data lakes. Specifically, LEDD can return semantically related tables based on natural-language specification. These features make LEDD an ideal foundation for downstream tasks such as model training and schema linking for text-to-SQL tasks. LEDD also provides a simple Python interface to facilitate the extension and the replacement of data discovery algorithms.