# Counterfactual Residual Data Augmentation for Regression

arXiv: https://arxiv.org/abs/2606.28460 · HTML: https://mlsanity.com/paper/2606.28460/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 26 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation](https://mlsanity.com/paper/2012.09092/) (2020)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [RegMix: Data Mixing Augmentation for Regression](https://mlsanity.com/paper/2106.03374/) (2021)
- [Cause-Effect Inference in Location-Scale Noise Models: Maximum Likelihood vs. Independence Testing](https://mlsanity.com/paper/2301.12930/) (2023)
- [Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounders](https://mlsanity.com/paper/2411.19923/) (2024)
- [When Shift Happens - Confounding Is to Blame](https://mlsanity.com/paper/2505.21422/) (2025)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)

## Built on by (most cited first, 1 of 1)

- [CAFE: Counterfactual Prediction via Fast Posterior Estimation](https://mlsanity.com/paper/2609.32167/) (2026)

## Abstract (arXiv)

Data-driven modeling in real-world regression tasks often suffers from limited training samples, high collection costs, and noisy observations. Inspired by the impact of data augmentation in vision and language, we propose a novel Counterfactual Residual Data Augmentation (CRDA) technique for tabular regression. Our key insight is that once a regressor has modeled the systematic component of the data, the remaining noise can be viewed as an invariant residual that remains stable under small perturbations of carefully selected features. We exploit this residual invariance to generate new, yet realistic, training samples, effectively expanding the dataset without requiring additional real data. Our method is model-agnostic and readily applicable to various types of regressors. In experiments across datasets from a variety of benchmark repositories, on average, CRDA reduces an MLP Regressor's MSE by 22.9% and an XGBoost Regressor's MSE by 6.4%. When compared to existing state-of-the-art data generators and augmentation techniques, CRDA consistently outperforms in MSE reduction. By adding principled counterfactual variations to the training data, our method offers a simple and efficient remedy for noise-prone, small-sample regression settings.