mlsanityData to 14 Oct 2026

A Context-Aware Approach for Enhancing Data Imputation with Pre-trained Language Models

arXiv 2405.17712 ↗First posted 28 May 2024cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel approach named Contextually Relevant Imputation leveraging pre-trained Language Models (CRILM) for handling missing data in tabular datasets. Instead of relying on traditional numerical estimations, CRILM uses pre-trained language models (LMs) to create contextually relevant descriptors for missing values. This method aligns datasets with LMs' strengths, allowing large LMs to generate these descriptors and small LMs to be fine-tuned on the enriched datasets for enhanced downstream task performance. Our evaluations demonstrate CRILM's superior performance and robustness across MCAR, MAR, and challenging MNAR scenarios, with up to a 10% improvement over the best-performing baselines. By mitigating biases, particularly in MNAR settings, CRILM improves downstream task performance and offers a cost-effective solution for resource-constrained environments.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 33 of its 77 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

33 of 77 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  9. 2015Importance Weighted Autoencoderscited by 456
  10. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  11. 2018GAIN: Missing Data Imputation using Generative Adversarial Netscited by 352
  12. 2018Handling Incomplete Heterogeneous Data using VAEscited by 149
21 more
  1. 2020Handling Missing Data with Graph Representation Learningcited by 133
  2. 2014Matrix Completion and Low-Rank SVD via Fast Alternating Least Squarescited by 104
  3. 2020Missing Data Imputation using Optimal Transportcited by 100
  4. 2022HyperImpute: Generalized Iterative Imputation with Automatic Model Selectioncited by 83
  5. 2019MisGAN: Learning from Incomplete Data with Generative Adversarial Networkscited by 75
  6. 2018Variational Autoencoder with Arbitrary Conditioningcited by 68
  7. 2018Graphical Models for Processing Missing Datacited by 58
  8. 2020MCFlow: Monte Carlo Flow Models for Data Imputationcited by 37
  9. 2023TabMT: Generating tabular data with masked transformerscited by 36
  10. 2023Transformed Distribution Matching for Missing Value Imputationcited by 31
  11. 2016Personalized Risk Scoring for Critical Care Prognosis using Mixtures of Gaussian Processescited by 19
  12. 2019Improving Missing Data Imputation with Deep Generative Modelscited by 19
  13. 2021Simultaneous Missing Value Imputation and Structure Learning with Groupscited by 16
  14. 2021Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problemscited by 15
  15. 2022Handling Missing Data via Max-Entropy Regularized Graph Autoencodercited by 12
  16. 2024Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretrainingcited by 12
  17. 2021EMFlow: Data Imputation in Latent Space via EM and Deep Flow Modelscited by 6
  18. 2022FragmGAN: Generative Adversarial Nets for Fragmentary Data Imputation and Predictioncited by 4
  19. 2020Multiple Imputation with Denoising Autoencoder using Metamorphic Truth and Imputation Feedbackcited by 3
  20. 2021A Graph-based Imputation Method for Sparse Medical Recordscited by 3
  21. 2022Multiple Imputation with Neural Network Gaussian Process for High-dimensional Incomplete Datacited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025A Survey of AIOps in the Era of Large Language Modelscited by 19
  2. 2026Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMscited by 9
  3. 2026Large Language Models for Missing Data Imputation: Understanding Behavior, Hallucination Effects, and Control Mechanismscited by 2
  4. 2025Precision Adaptive Imputation Network : An Unified Technique for Mixed Datasetscited by 0
  5. 2025An Interdisciplinary and Cross-Task Review on Missing Data Imputationcited by 0