# Noise Regularization for Conditional Density Estimation

arXiv: https://arxiv.org/abs/1907.08982 · HTML: https://mlsanity.com/paper/1907.08982/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2019.

## Built on by (most cited first, 23 of 23)

- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [Meta-Learning Reliable Priors in the Function Space](https://mlsanity.com/paper/2106.03195/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Normalizing Flows for Interventional Density Estimation](https://mlsanity.com/paper/2209.06203/) (2022)
- [Bounds on Representation-Induced Confounding Bias for Treatment Effect Estimation](https://mlsanity.com/paper/2311.11321/) (2023)
- [IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning](https://mlsanity.com/paper/2302.03271/) (2023)
- [Multi-Asset Spot and Option Market Simulation](https://mlsanity.com/paper/2112.06823/) (2021)
- [Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model](https://mlsanity.com/paper/2306.01424/) (2023)
- [IL-flOw: Imitation Learning from Observation using Normalizing Flows](https://mlsanity.com/paper/2205.09251/) (2022)
- [Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner](https://mlsanity.com/paper/2411.03387/) (2024)
- [Anomaly Detection with Variance Stabilized Density Estimation](https://mlsanity.com/paper/2306.00582/) (2023)
- [Short-Term Density Forecasting of Low-Voltage Load using Bernstein-Polynomial Normalizing Flows](https://mlsanity.com/paper/2204.13939/) (2022)
- [MGF: Mixed Gaussian Flow for Diverse Trajectory Prediction](https://mlsanity.com/paper/2402.12238/) (2024)
- [Conditional Density Estimation with Histogram Trees](https://mlsanity.com/paper/2410.11449/) (2024)
- [Deconvolutional Density Network: Modeling Free-Form Conditional Distributions](https://mlsanity.com/paper/2105.14367/) (2021)
- [Uncertainty estimation under model misspecification in neural network regression](https://mlsanity.com/paper/2111.11763/) (2021)
- [Variational Autoencoder Framework for Hyperspectral Retrievals (Hyper-VAE) of Phytoplankton Absorption and Chlorophyll a in Coastal Waters for NASA's EMIT and PACE Missions](https://mlsanity.com/paper/2504.13476/) (2025)
- [Contrastive Representation Learning with Trainable Augmentation Channel](https://mlsanity.com/paper/2111.07679/) (2021)
- [GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes](https://mlsanity.com/paper/2509.22953/) (2025)
- [Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimation](https://mlsanity.com/paper/2509.24962/) (2025)
- [Nonparametric estimation of conditional probability distributions using a generative approach based on conditional push-forward neural networks](https://mlsanity.com/paper/2511.14455/) (2025)
- [Predictability Analysis of Regression Problems via Conditional Entropy Estimations](https://mlsanity.com/paper/2406.03824/) (2024)
- [CEC-MMR: Cross-Entropy Clustering Approach to Multi-Modal Regression](https://mlsanity.com/paper/2504.07301/) (2025)

## Abstract (arXiv)

Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum likelihood objective. Due to the inherent structure of such models, classical regularization approaches in the parameter space are rendered ineffective. To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training. We demonstrate that the proposed approach corresponds to a smoothness regularization and prove its asymptotic consistency. In our experiments, noise regularization significantly and consistently outperforms other regularization methods across seven data sets and three CDE models. The effectiveness of noise regularization makes neural network based CDE the preferable method over previous non- and semi-parametric approaches, even when training data is scarce.