# Neural-g: A Deep Learning Framework for Mixing Density Estimation

arXiv: https://arxiv.org/abs/2406.05986 · HTML: https://mlsanity.com/paper/2406.05986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation](https://mlsanity.com/paper/1704.08095/) (2017)
- [Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference](https://mlsanity.com/paper/2011.05836/) (2020)
- [Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models](https://mlsanity.com/paper/2008.08244/) (2020)
- [Two Modeling Strategies for Empirical Bayes Estimation](https://mlsanity.com/paper/1409.2677/) (2014)
- [On Approximation Capabilities of ReLU Activation and Softmax Output Layer in Neural Networks](https://mlsanity.com/paper/2002.04060/) (2020)
- [Density Deconvolution with Normalizing Flows](https://mlsanity.com/paper/2006.09396/) (2020)
- [Generative Quantile Regression with Variability Penalty](https://mlsanity.com/paper/2301.03661/) (2023)
- [A Nonparametric Maximum Likelihood Approach to Mixture of Regression](https://mlsanity.com/paper/2108.09816/) (2021)
- [Boosting in Univariate Nonparametric Maximum Likelihood Estimation](https://mlsanity.com/paper/2101.08505/) (2021)
- [Fast Bootstrapping Nonparametric Maximum Likelihood for Latent Mixture Models](https://mlsanity.com/paper/2402.18748/) (2024)

## Abstract (arXiv)

Mixing (or prior) density estimation is an important problem in machine learning and statistics, especially in empirical Bayes g-modeling where accurately estimating the prior is necessary for making good posterior inferences. In this paper, we propose neural-g, a new neural network-based estimator for g-modeling. Neural-g uses a softmax output layer to ensure that the estimated prior is a valid probability density. Under default hyperparameters, we show that neural-g is very flexible and capable of capturing many unknown densities, including those with flat regions, heavy tails, and/or discontinuities. In contrast, existing methods struggle to capture all of these prior shapes. We provide justification for neural-g by establishing a new universal approximation theorem regarding the capability of neural networks to learn arbitrary probability mass functions. To accelerate convergence of our numerical implementation, we utilize a weighted average gradient descent approach to update the network parameters. Finally, we extend neural-g to multivariate prior density estimation. We illustrate the efficacy of our approach through simulations and analyses of real datasets. A software package to implement neural-g is publicly available at https://github.com/shijiew97/neuralG.