# Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation

arXiv: https://arxiv.org/abs/1704.08095 · HTML: https://mlsanity.com/paper/1704.08095/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 30 of 34)

- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [Wasserstein Generative Learning of Conditional Distribution](https://mlsanity.com/paper/2112.10039/) (2021)
- [Split Localized Conformal Prediction](https://mlsanity.com/paper/2206.13092/) (2022)
- [Model Based Explanations of Concept Drift](https://mlsanity.com/paper/2303.09331/) (2023)
- [Neural Conditional Probability for Uncertainty Quantification](https://mlsanity.com/paper/2407.01171/) (2024)
- [Towards Instance-Wise Calibration: Local Amortized Diagnostics and Reshaping of Conditional Densities (LADaR)](https://mlsanity.com/paper/2205.14568/) (2022)
- [Volume Optimality in Conformal Prediction with Structured Prediction Sets](https://mlsanity.com/paper/2502.16658/) (2025)
- [Explaining predictive models using Shapley values and non-parametric vine copulas](https://mlsanity.com/paper/2102.06416/) (2021)
- [Epistemic Uncertainty in Conformal Scores: A Unified Approach](https://mlsanity.com/paper/2502.06995/) (2025)
- [A unified framework for dataset shift diagnostics](https://mlsanity.com/paper/2205.08340/) (2022)
- [Conditional independence testing under misspecified inductive biases](https://mlsanity.com/paper/2307.02520/) (2023)
- [One or Two Things We know about Concept Drift -- A Survey on Monitoring Evolving Environments](https://mlsanity.com/paper/2310.15826/) (2023)
- [MD-split+: Practical Local Conformal Inference in High Dimensions](https://mlsanity.com/paper/2107.03280/) (2021)
- [Conformalized Deep Splines for Optimal and Efficient Prediction Sets](https://mlsanity.com/paper/2311.00774/) (2023)
- [shapr: Explaining Machine Learning Models with Conditional Shapley Values in R and Python](https://mlsanity.com/paper/2504.01842/) (2025)
- [A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models](https://mlsanity.com/paper/2410.02025/) (2024)
- [Re-calibrating Photometric Redshift Probability Distributions Using Feature-space Regression](https://mlsanity.com/paper/2110.15209/) (2021)
- [Schr"odinger bridge based deep conditional generative learning](https://mlsanity.com/paper/2409.17294/) (2024)
- [Hierarchical clustering: visualization, feature importance and model selection](https://mlsanity.com/paper/2112.01372/) (2021)
- [Conformal inference for regression on Riemannian Manifolds](https://mlsanity.com/paper/2310.08209/) (2023)
- [Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task](https://mlsanity.com/paper/2511.18530/) (2025)
- [Generative and Nonparametric Approaches for Conditional Distribution Estimation: Methods, Perspectives, and Comparative Evaluations](https://mlsanity.com/paper/2601.22650/) (2026)
- [Continuous Treatment Recommendation with Deep Survival Dose Response Function](https://mlsanity.com/paper/2108.10453/) (2021)
- [MCD: Marginal Contrastive Discrimination for conditional density estimation](https://mlsanity.com/paper/2206.01592/) (2022)
- [Flexible conditional density estimation for time series](https://mlsanity.com/paper/2301.09671/) (2023)
- [Kernel Density Bayesian Inverse Reinforcement Learning](https://mlsanity.com/paper/2303.06827/) (2023)
- [Detecting Errors in a Numerical Response via any Regression Model](https://mlsanity.com/paper/2305.16583/) (2023)
- [High-Dimensional Tail Index Regression](https://mlsanity.com/paper/2403.01318/) (2024)
- [Neural-g: A Deep Learning Framework for Mixing Density Estimation](https://mlsanity.com/paper/2406.05986/) (2024)
- [Non-parametric efficient estimation of marginal structural models with multi-valued time-varying treatments](https://mlsanity.com/paper/2409.18782/) (2024)

## Abstract (arXiv)

There is a growing demand for nonparametric conditional density estimators (CDEs) in fields such as astronomy and economics. In astronomy, for example, one can dramatically improve estimates of the parameters that dictate the evolution of the Universe by working with full conditional densities instead of regression (i.e., conditional mean) estimates. More generally, standard regression falls short in any prediction problem where the distribution of the response is more complex with multi-modality, asymmetry or heteroscedastic noise. Nevertheless, much of the work on high-dimensional inference concerns regression and classification only, whereas research on density estimation has lagged behind. Here we propose FlexCode, a fully nonparametric approach to conditional density estimation that reformulates CDE as a non-parametric orthogonal series problem where the expansion coefficients are estimated by regression. By taking such an approach, one can efficiently estimate conditional densities and not just expectations in high dimensions by drawing upon the success in high-dimensional regression. Depending on the choice of regression procedure, our method can adapt to a variety of challenging high-dimensional settings with different structures in the data (e.g., a large number of irrelevant components and nonlinear manifold structure) as well as different data types (e.g., functional data, mixed data types and sample sets). We study the theoretical and empirical performance of our proposed method, and we compare our approach with traditional conditional density estimators on simulated as well as real-world data, such as photometric galaxy data, Twitter data, and line-of-sight velocities in a galaxy cluster.