# In-Database Data Imputation

arXiv: https://arxiv.org/abs/2401.03359 · HTML: https://mlsanity.com/paper/2401.03359/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 79 references that are in the map.
- First posted on arXiv 7 Jan 2024.

## Builds on (most cited first, 20 of 20)

- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Handling Missing Data with Graph Representation Learning](https://mlsanity.com/paper/2010.16418/) (2020)
- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [Diffusion models for missing value imputation in tabular data](https://mlsanity.com/paper/2210.17128/) (2022)
- [MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms](https://mlsanity.com/paper/2111.03187/) (2021)
- [Missing Value Imputation on Multidimensional Time Series](https://mlsanity.com/paper/2103.01600/) (2021)
- [Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions](https://mlsanity.com/paper/2005.05117/) (2020)
- [Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo](https://mlsanity.com/paper/2202.04599/) (2022)
- [Towards Linear Algebra over Normalized Data](https://mlsanity.com/paper/1612.07448/) (2016)
- [A Layered Aggregate Engine for Analytics Workloads](https://mlsanity.com/paper/1906.08687/) (2019)
- [AC/DC: In-Database Learning Thunderstruck](https://mlsanity.com/paper/1803.07480/) (2018)
- [Learning Over Dirty Data Without Cleaning](https://mlsanity.com/paper/2004.02308/) (2020)
- [JoinBoost: Grow Trees Over Normalized Data Using Only SQL](https://mlsanity.com/paper/2307.00422/) (2023)
- [F-IVM: Learning over Fast-Evolving Relational Data](https://mlsanity.com/paper/2006.00694/) (2020)
- [LMFAO: An Engine for Batches of Group-By Aggregates](https://mlsanity.com/paper/2008.08657/) (2020)
- [ReStore -- Neural Data Completion for Relational Databases](https://mlsanity.com/paper/2105.12457/) (2021)
- [Fast and Reliable Missing Data Contingency Analysis with Predicate-Constraints](https://mlsanity.com/paper/2004.04139/) (2020)
- [Online Topic-Aware Entity Resolution Over Incomplete Data Streams (Technical Report)](https://mlsanity.com/paper/2103.08720/) (2021)

## Built on by (most cited first, 3 of 3)

- [Precision Adaptive Imputation Network : An Unified Technique for Mixed Datasets](https://mlsanity.com/paper/2501.10667/) (2025)
- [Learning Over Dirty Data with Minimal Repairs](https://mlsanity.com/paper/2503.13921/) (2025)
- [Imputation of Missing Data in Smooth Pursuit Eye Movements Using a Self-Attention-based Deep Learning Approach](https://mlsanity.com/paper/2506.00545/) (2025)

## Abstract (arXiv)

Missing data is a widespread problem in many domains, creating challenges in data analysis and decision making. Traditional techniques for dealing with missing data, such as excluding incomplete records or imputing simple estimates (e.g., mean), are computationally efficient but may introduce bias and disrupt variable relationships, leading to inaccurate analyses. Model-based imputation techniques offer a more robust solution that preserves the variability and relationships in the data, but they demand significantly more computation time, limiting their applicability to small datasets. This work enables efficient, high-quality, and scalable data imputation within a database system using the widely used MICE method. We adapt this method to exploit computation sharing and a ring abstraction for faster model training. To impute both continuous and categorical values, we develop techniques for in-database learning of stochastic linear regression and Gaussian discriminant analysis models. Our MICE implementations in PostgreSQL and DuckDB outperform alternative MICE implementations and model-based imputation techniques by up to two orders of magnitude in terms of computation time, while maintaining high imputation quality.