mlsanityData to 7 Oct 2026

Deep Data Density Estimation through Donsker-Varadhan Representation

arXiv 2104.06612 ↗First posted 14 Apr 2021cs.LG cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Estimating the data density is one of the challenging problems in deep learning. In this paper, we present a simple yet effective method for estimating the data density using a deep neural network and the Donsker-Varadhan variational lower bound on the KL divergence. We show that the optimal critic function associated with the Donsker-Varadhan representation on the KL divergence between the data and the uniform distribution can estimate the data density. We also present the deep neural network-based modeling and its stochastic learning. The experimental results and possible applications of the proposed method demonstrate that it is competitive with the previous methods and has a lot of possibilities in applied to various applications.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

25 of 39 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  8. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  9. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  10. 2016Deep Variational Information Bottleneckcited by 915
  11. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  12. 2016Pixel Recurrent Neural Networkscited by 814
13 more
  1. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  2. 2018MINE: Mutual Information Neural Estimationcited by 680
  3. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  4. 2019On Variational Bounds of Mutual Informationcited by 550
  5. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  6. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  7. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  8. 2019On Mutual Information Maximization for Representation Learningcited by 315
  9. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  10. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  11. 2013High-Dimensional Probability Estimation with Deep Density Modelscited by 17
  12. 2020Interpreting Rate-Distortion of Variational Autoencoder and Using Model Uncertainty for Anomaly Detectioncited by 10
  13. 2019Combining Stochastic Adaptive Cubic Regularization with Negative Curvature for Nonconvex Optimizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2022Confidence-Aware Graph Neural Networks for Learning Reliability Assessment Commitmentscited by 12
  2. 2025Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learningcited by 4
  3. 2024REMEDI: Corrective Transformations for Improved Neural Entropy Estimationcited by 2
  4. 2024TREET: TRansfer Entropy Estimation via Transformerscited by 2
  5. 2023Conformalised data synthesiscited by 0
  6. 2026NMINE: Normalized Mutual Information Neural Estimationcited by 0