mlsanityData to 14 Oct 2026

Empowering Tabular Data Preparation with Language Models: Why and How?

arXiv 2508.01556 ↗First posted 3 Aug 2025cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Data preparation is a critical step in enhancing the usability of tabular data and thus boosts downstream data-driven tasks. Traditional methods often face challenges in capturing the intricate relationships within tables and adapting to the tasks involved. Recent advances in Language Models (LMs), especially in Large Language Models (LLMs), offer new opportunities to automate and support tabular data preparation. However, why LMs suit tabular data preparation (i.e., how their capabilities match task demands) and how to use them effectively across phases still remain to be systematically explored. In this survey, we systematically analyze the role of LMs in enhancing tabular data preparation processes, focusing on four core phases: data acquisition, integration, cleaning, and transformation. For each phase, we present an integrated analysis of how LMs can be combined with other components for different preparation tasks, highlight key advancements, and outline prospective pipelines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 50 of its 102 references that are in the map. First posted on arXiv 3 Aug 2025.

Builds on

50 of 102 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2017Billion-scale similarity search with GPUscited by 2,497
  10. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  11. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  12. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
38 more
  1. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  2. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  3. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 144
  4. 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 111
  5. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
  6. 2021Open Domain Question Answering over Tables via Dense Retrievalcited by 74
  7. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  8. 2024Large Language Model for Table Processing: A Surveycited by 52
  9. 2023Open-WikiTable: Dataset for Open Domain Question Answering with Complex Reasoning over Tablecited by 40
  10. 2022Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learningcited by 38
  11. 2020Valentine: Evaluating Matching Techniques for Dataset Discoverycited by 35
  12. 2023Large Language Models as Data Preprocessorscited by 34
  13. 2024Match, Compare, or Select? An Investigation of Large Language Models for Entity Matchingcited by 32
  14. 2016LSH Ensemble: Internet-Scale Domain Searchcited by 24
  15. 2021Retrieving Complex Tables with Multi-Granular Graph Representation Learningcited by 24
  16. 2019AlphaClean: Automatic Generation of Data Cleaning Pipelinescited by 23
  17. 2020(Almost) All of Entity Resolutioncited by 23
  18. 2020Table Search Using a Deep Contextualized Language Modelcited by 22
  19. 2025Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunitiescited by 22
  20. 2024Schema Matching with Large Language Models: an Experimental Studycited by 20
  21. 2024Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performancecited by 20
  22. 2024Magneto: Combining Small and Large Language Models for Schema Matchingcited by 17
  23. 2024ReMatch: Retrieval Enhanced Schema Matching with LLMscited by 16
  24. 2024Fine-tuning Large Language Models for Entity Matchingcited by 16
  25. 2023Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examplescited by 15
  26. 2025Knowledge Graph-based Retrieval-Augmented Generation for Schema Matchingcited by 13
  27. 2024Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representationscited by 12
  28. 2023Solo: Data Discovery Using Natural Language Questions Via A Self-Supervised Approachcited by 11
  29. 2025On LLM-Enhanced Mixed-Type Data Imputation with High-Order Message Passingcited by 11
  30. 2024Matchmaker: Self-Improving Large Language Model Programs for Schema Matchingcited by 9
  31. 2023Automatic Data Transformation Using Large Language Model: An Experimental Study on Building Energy Datacited by 8
  32. 2024KcMF: A Knowledge-compliant Framework for Schema and Entity Matching with Fine-tuning-free LLMscited by 8
  33. 2025General Table Question Answering via Answer-Formula Joint Generationcited by 7
  34. 2024BoostER: Leveraging Large Language Models for Enhancing Entity Resolutioncited by 6
  35. 2025Birdie: Natural Language-Driven Table Discovery Using Differentiable Search Indexcited by 5
  36. 2025Text-to-Pipeline: Bridging Natural Language and Data Preparation Pipelinescited by 5
  37. 2025Snoopy: Effective and Efficient Semantic Join Discovery via Proxy Columnscited by 4
  38. 2024OmniMatch: Effective Self-Supervised Any-Join Discovery in Tabular Data Repositoriescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMscited by 9
  2. 2026PrepBench: How Far Are We from Natural-Language-Driven Data Preparation?cited by 3
  3. 2026Interpretable Column Annotation with LLM-Symbolized Decision Process Materializationcited by 0
  4. 2026Open Tabular Insight Extraction: Where Do We Stand, and Where Should We Go?cited by 0