mlsanityData to 7 Oct 2026

Can Foundation Models Wrangle Your Data?

arXiv 2205.09911 ↗First posted 20 May 2022cs.LG cs.AI cs.DBcited by 173

Not on the map. Why?

Abstract (arXiv)

Foundation Models (FMs) are models trained on large corpora of data that, at very large scale, can generalize to new tasks without any task-specific finetuning. As these models continue to grow in size, innovations continue to push the boundaries of what these models can do on language and image tasks. This paper aims to understand an underexplored area of FMs: classical data tasks like cleaning and integration. As a proof-of-concept, we cast five data cleaning and integration tasks as prompting tasks and evaluate the performance of FMs on these tasks. We find that large FMs generalize and achieve SoTA performance on data cleaning and integration tasks, even though they are not trained for these data tasks. We identify specific research challenges and opportunities that these models present, including challenges with private and domain specific data, and opportunities to make data management systems more accessible to non-experts. We make our code and experiments publicly available at: https://github.com/HazyResearch/fm_data_tasks.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). Builds on 49 of its 103 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

49 of 103 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
37 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  13. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  14. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  15. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  16. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  17. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  18. 2020Jukebox: A Generative Model for Musiccited by 535
  19. 2021Fast Model Editing at Scalecited by 491
  20. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  21. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  22. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  23. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  24. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  25. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  26. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  27. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
  28. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 143
  29. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  30. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  31. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  32. 2017HoloClean: Holistic Data Repairs with Probabilistic Inferencecited by 79
  33. 2022Standing on the Shoulders of Giant Frozen Language Modelscited by 33
  34. 2019HoloDetect: Few-Shot Learning for Error Detectioncited by 31
  35. 2022Reasoning over Public and Private Data in Retrieval-Based Systemscited by 21
  36. 2020Conditioned Natural Language Generation using only Unconditioned Language Model: An Explorationcited by 12
  37. 2021Language Models As or For Knowledge Basescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 44; 2024: 47; 2025: 45; 2026: 33.

173 papers.

Sort:

By area:

  1. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  2. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  3. 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 167
  4. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  5. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  6. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  7. 2023Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakescited by 101
  8. 2023S3: Increasing GPU Utilization during Generative Inference for Higher Throughputcited by 81
  9. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUscited by 73
  10. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  11. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 69
  12. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Modelscited by 64
38 more
  1. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
  2. 2024Queue management for slo-oriented large language model servingcited by 36
  3. 2023Large Language Models as Data Preprocessorscited by 34
  4. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  5. 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
  6. 2023Entity Matching using Large Language Modelscited by 31
  7. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  8. 2023TABLET: Learning From Instructions For Tabular Datacited by 28
  9. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 27
  10. 2023CHORUS: Foundation Models for Unified Data Discovery and Explorationcited by 24
  11. 2023Using ChatGPT for Entity Matchingcited by 22
  12. 2023Language models are weak learnerscited by 21
  13. 2023Jellyfish: A Large Language Model for Data Preprocessingcited by 20
  14. 2024The Design of an LLM-powered Unstructured Analytics Systemcited by 20
  15. 2025Exploring LLM Agents for Cleaning Tabular Machine Learning Datasetscited by 20
  16. 2024SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelinescited by 19
  17. 2024Elephants Never Forget: Testing Language Models for Memorization of Tabular Datacited by 19
  18. 2024Schema Matching with Large Language Models: an Experimental Studycited by 19
  19. 2023Column Type Annotation using ChatGPTcited by 18
  20. 2023Improving Retrieval-Augmented Large Language Models via Data Importance Learningcited by 17
  21. 2024Magneto: Combining Small and Large Language Models for Schema Matchingcited by 17
  22. 2024ReMatch: Retrieval Enhanced Schema Matching with LLMscited by 16
  23. 2024NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalizationcited by 16
  24. 2024Fine-tuning Large Language Models for Entity Matchingcited by 16
  25. 2025BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documentscited by 16
  26. 2022DeepJoin: Joinable Table Discovery with Pre-trained Language Modelscited by 15
  27. 2023Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examplescited by 15
  28. 2023ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Modelscited by 15
  29. 2023Cost-Effective In-Context Learning for Entity Resolution: A Design Space Explorationcited by 15
  30. 2025Seesaw: High-throughput LLM Inference via Model Re-shardingcited by 15
  31. 2023Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsitycited by 14
  32. 2023InstructExcel: A Benchmark for Natural Language Instruction in Excelcited by 14
  33. 2023A Hardware Evaluation Framework for Large Language Model Inferencecited by 14
  34. 2024Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualizationcited by 14
  35. 2024Automating the Enterprise with Foundation Modelscited by 14
  36. 2023Querying Large Language Models with SQLcited by 13
  37. 2023One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Datacited by 12
  38. 2023Construction of Knowledge Graphs: State and Challengescited by 12

50 most cited of 173 All 173 (JSON)