# PrepBench: How Far Are We from Natural-Language-Driven Data Preparation? arXiv: https://arxiv.org/abs/2605.08687 ยท HTML: https://mlsanity.com/paper/2605.08687/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 38 of its 59 references that are in the map. - First posted on arXiv 9 May 2026. ## Builds on (most cited first, 30 of 38) - [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018) - [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023) - [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022) - [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019) - [MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://mlsanity.com/paper/2309.10691/) (2023) - [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024) - [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024) - [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022) - [InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks](https://mlsanity.com/paper/2401.05507/) (2024) - [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023) - [SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation](https://mlsanity.com/paper/2406.14991/) (2024) - [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022) - [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017) - [SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models](https://mlsanity.com/paper/2305.19308/) (2023) - [DABstep: Data Agent Benchmark for Multi-step Reasoning](https://mlsanity.com/paper/2506.23719/) (2025) - [Large Language Models as Data Preprocessors](https://mlsanity.com/paper/2308.16361/) (2023) - [KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes](https://mlsanity.com/paper/2506.06541/) (2025) - [Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching](https://mlsanity.com/paper/2405.16884/) (2024) - [Benchmarking Data Science Agents](https://mlsanity.com/paper/2402.17168/) (2024) - [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025) - [TempTabQA: Temporal Question Answering for Semi-Structured Tables](https://mlsanity.com/paper/2311.08002/) (2023) - [Agent-OM: Leveraging LLM Agents for Ontology Matching](https://mlsanity.com/paper/2312.00326/) (2023) - [CHORUS: Foundation Models for Unified Data Discovery and Exploration](https://mlsanity.com/paper/2306.09610/) (2023) - [AgenticData: An Agentic Data Analytics System for Heterogeneous Data](https://mlsanity.com/paper/2508.05002/) (2025) - [Schema Matching with Large Language Models: an Experimental Study](https://mlsanity.com/paper/2407.11852/) (2024) - [PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries](https://mlsanity.com/paper/2410.11076/) (2024) - [BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactions](https://mlsanity.com/paper/2510.05318/) (2025) - [SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models](https://mlsanity.com/paper/2403.03636/) (2024) - [ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines](https://mlsanity.com/paper/2504.04808/) (2025) - [AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework](https://mlsanity.com/paper/2412.10422/) (2024) ## Built on by (most cited first, 3 of 3) - [AgenticDataBench: A Comprehensive Benchmark for Data Agents](https://mlsanity.com/paper/2607.01647/) (2026) - [Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows](https://mlsanity.com/paper/2607.07504/) (2026) - [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026) ## Abstract (arXiv) Data preparation is a central and time-consuming stage in data analysis workflows. Traditionally, commercial tools have relied on graphical user interfaces (GUIs) to simplify data preparation, allowing users to define transformations through visual operators and workflows. Recent advances in large language models (LLMs) raise the possibility of a paradigm shift toward natural language (NL)-driven data preparation, in which users can specify preparation intents in NL directly. However, it remains unclear how far current LLM-based agents are from this paradigm shift in practice. Existing code generation benchmarks do not capture key characteristics of data preparation, including ambiguous user intents, imperfect real-world data, and the need to translate code into interpretable workflows for validation. To bridge this gap, we present PrepBench, a benchmark designed to evaluate NL-driven data preparation along three core capabilities: interactive disambiguation, prep-code generation, and code-to-workflow translation. We crawl data from the Preppin' Data Challenges, and then extend it into a systematically designed benchmark. The benchmark covers diverse domains, and each task involves 3 to 18 data preparation steps. Nearly half of the tasks require over 100 lines of Python code, and the longest solutions approach 300 lines. Our evaluation shows that, despite recent progress, realizing this paradigm shift remains challenging for state-of-the-art LLMs. PrepBench provides a principled benchmark for measuring this gap and helps identify key challenges toward realizing NL-driven data preparation.