# C-Mixup: Improving Generalization in Regression

arXiv: https://arxiv.org/abs/2210.05775 · HTML: https://mlsanity.com/paper/2210.05775/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 90 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 30 of 58)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Gradient Descent Maximizes the Margin of Homogeneous Neural Networks](https://mlsanity.com/paper/1906.05890/) (2019)
- [Data Augmentation Generative Adversarial Networks](https://mlsanity.com/paper/1711.04340/) (2017)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2104.09937/) (2021)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data](https://mlsanity.com/paper/1909.00889/) (2019)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Deep Domain-Adversarial Image Generation for Domain Generalisation](https://mlsanity.com/paper/2003.06054/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [MixUp as Locally Linear Out-Of-Manifold Regularization](https://mlsanity.com/paper/1809.02499/) (2018)

## Built on by (most cited first, 30 of 57)

- [Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise](https://mlsanity.com/paper/2210.11075/) (2022)
- [Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift](https://mlsanity.com/paper/2306.12306/) (2023)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [HappyMap: A Generalized Multi-calibration Method](https://mlsanity.com/paper/2303.04379/) (2023)
- [MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation](https://mlsanity.com/paper/2307.07832/) (2023)
- [Improving Domain Generalization with Domain Relations](https://mlsanity.com/paper/2302.02609/) (2023)
- [Semi-Supervised Graph Imbalanced Regression](https://mlsanity.com/paper/2305.12087/) (2023)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation](https://mlsanity.com/paper/2311.11961/) (2023)
- [Generalizing to Unseen Domains in Diabetic Retinopathy with Disentangled Representations](https://mlsanity.com/paper/2406.06384/) (2024)
- [Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift](https://mlsanity.com/paper/2406.00661/) (2024)
- [R-Mixup: Riemannian Mixup for Biological Networks](https://mlsanity.com/paper/2306.02532/) (2023)
- [Switch EMA: A Free Lunch for Better Flatness and Sharpness](https://mlsanity.com/paper/2402.09240/) (2024)
- [Data Augmentation via Subgroup Mixup for Improving Fairness](https://mlsanity.com/paper/2309.07110/) (2023)
- [MOSPC: MOS Prediction Based on Pairwise Comparison](https://mlsanity.com/paper/2306.10493/) (2023)
- [Anchor Data Augmentation](https://mlsanity.com/paper/2311.06965/) (2023)
- [RC-Mixup: A Data Augmentation Strategy against Noisy Data for Regression Tasks](https://mlsanity.com/paper/2405.17938/) (2024)
- [First-Order Manifold Data Augmentation for Regression Learning](https://mlsanity.com/paper/2406.10914/) (2024)
- [Out-Of-Distribution Detection with Diversification (Provably)](https://mlsanity.com/paper/2411.14049/) (2024)
- [Tailoring Mixup to Data for Calibration](https://mlsanity.com/paper/2311.01434/) (2023)
- [StarLKNet: Star Mixup with Large Kernel Networks for Palm Vein Identification](https://mlsanity.com/paper/2405.12721/) (2024)
- [Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learning](https://mlsanity.com/paper/2410.13792/) (2024)
- [Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks](https://mlsanity.com/paper/2306.17630/) (2023)
- [ADASSM: Adversarial Data Augmentation in Statistical Shape Models From Images](https://mlsanity.com/paper/2307.03273/) (2023)
- [Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory](https://mlsanity.com/paper/2308.02966/) (2023)
- [Sample Selection via Contrastive Fragmentation for Noisy Label Regression](https://mlsanity.com/paper/2502.17771/) (2025)
- [Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations](https://mlsanity.com/paper/2210.14358/) (2022)
- [SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixup](https://mlsanity.com/paper/2309.16633/) (2023)
- [Augment on Manifold: Mixup Regularization with UMAP](https://mlsanity.com/paper/2312.13141/) (2023)
- [Uncertainty-Guided Alignment for Unsupervised Domain Adaptation in Regression](https://mlsanity.com/paper/2401.13721/) (2024)

## Abstract (arXiv)

Improving the generalization of deep networks is an important open challenge, particularly in domains without plentiful data. The mixup algorithm improves generalization by linearly interpolating a pair of examples and their corresponding labels. These interpolated examples augment the original training set. Mixup has shown promising results in various classification tasks, but systematic analysis of mixup in regression remains underexplored. Using mixup directly on regression labels can result in arbitrarily incorrect labels. In this paper, we propose a simple yet powerful algorithm, C-Mixup, to improve generalization on regression tasks. In contrast with vanilla mixup, which picks training examples for mixing with uniform probability, C-Mixup adjusts the sampling probability based on the similarity of the labels. Our theoretical analysis confirms that C-Mixup with label similarity obtains a smaller mean square error in supervised regression and meta-regression than vanilla mixup and using feature similarity. Another benefit of C-Mixup is that it can improve out-of-distribution robustness, where the test distribution is different from the training distribution. By selectively interpolating examples with similar labels, it mitigates the effects of domain-associated information and yields domain-invariant representations. We evaluate C-Mixup on eleven datasets, ranging from tabular to video data. Compared to the best prior approach, C-Mixup achieves 6.56%, 4.76%, 5.82% improvements in in-distribution generalization, task generalization, and out-of-distribution robustness, respectively. Code is released at https://github.com/huaxiuyao/C-Mixup.