mlsanityData to 7 Oct 2026

Neural Variational Inference and Learning in Belief Networks

arXiv 1402.0030 ↗First posted 31 Jan 2014cs.LG stat.MLcited by 116

Not on the map. Why?

Abstract (arXiv)

Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well. We propose a fast non-iterative approximate inference method that uses a feedforward network to implement efficient exact sampling from the variational posterior. The model and this inference network are trained jointly by maximizing a variational lower bound on the log-likelihood. Although the naive estimator of the inference model gradient is too high-variance to be useful, we make it practical by applying several straightforward model-independent variance reduction techniques. Applying our approach to training sigmoid belief networks and deep autoregressive networks, we show that it outperforms the wake-sleep algorithm on MNIST and achieves state-of-the-art results on the Reuters RCV1 document dataset.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jan 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 34; 2022: 31; 2023: 17; 2024: 13; 2025: 12; 2026: 8.

116 papers.

Sort:

By area:

  1. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  2. 2022GFlowNets and variational inferencecited by 103
  3. 2022Tutorial on amortized optimizationcited by 83
  4. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  5. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  6. 2021Learning Semantic Segmentation of Large-Scale Point Clouds with Random Samplingcited by 51
  7. 2021MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networkscited by 44
  8. 2024A safety realignment framework via subspace-oriented model fusion for large language modelscited by 40
  9. 2022Recent Advances for Quantum Neural Networks in Generative Learningcited by 39
  10. 2024Neural Methods for Amortized Inferencecited by 36
  11. 2022AdaProp: Learning Adaptive Propagation for Graph Neural Network based Knowledge Graph Reasoningcited by 28
  12. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
38 more
  1. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
  2. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 26
  3. 2021FoveaTer: Foveated Transformer for Image Classificationcited by 26
  4. 2021Learning Opinion Summarizers by Selecting Informative Reviewscited by 25
  5. 2022Gradient Estimation with Discrete Stein Operatorscited by 22
  6. 2022Sparse Graph Learning from Spatiotemporal Time Seriescited by 22
  7. 2022Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flowscited by 22
  8. 2023Estimating Conditional Mutual Information for Dynamic Feature Selectioncited by 20
  9. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  10. 2021Guiding Visual Question Generationcited by 14
  11. 2023Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trickcited by 13
  12. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  13. 2021Storchastic: A Framework for General Stochastic Automatic Differentiationcited by 11
  14. 2021GenNI: Human-AI Collaboration for Data-Backed Text Generationcited by 11
  15. 2021Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpointscited by 11
  16. 2023Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segmentscited by 11
  17. 2024Neuro-Symbolic AI: Explainability, Challenges, and Future Trendscited by 10
  18. 2021Coupled Gradient Estimators for Discrete Latent Variablescited by 9
  19. 2023Latent Class-Conditional Noise Modelcited by 9
  20. 2021ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variablescited by 8
  21. 2021Unsupervised Learning of Neurosymbolic Encoderscited by 8
  22. 2021Toward a `Standard Model' of Machine Learningcited by 8
  23. 2021Double Control Variates for Gradient Estimation in Discrete Latent Variable Modelscited by 8
  24. 2022Autoregressive Co-Training for Learning Discrete Speech Representationscited by 8
  25. 2022A Variational Approach to Bayesian Phylogenetic Inferencecited by 8
  26. 2022Training Discrete Deep Generative Models via Gapped Straight-Through Estimatorcited by 8
  27. 2022Compressed Vision for Efficient Video Understandingcited by 8
  28. 2021Cross-Modal Generative Augmentation for Visual Question Answeringcited by 7
  29. 2025Causal Time Series Generation via Diffusion Modelscited by 7
  30. 2021Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Modelscited by 6
  31. 2022Foundation Posteriors for Approximate Probabilistic Inferencecited by 6
  32. 2023Block-local learning with probabilistic latent representationscited by 6
  33. 2021Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysiscited by 5
  34. 2022Drawing out of Distribution with Neuro-Symbolic Generative Modelscited by 5
  35. 2022INTERACTION: A Generative XAI Framework for Natural Language Inference Explanationscited by 5
  36. 2022Multi-Sample Training for Neural Image Compressioncited by 5
  37. 2022Variational Open-Domain Question Answeringcited by 5
  38. 2025Multilevel Generative Samplers for Investigating Critical Phenomenacited by 5

50 most cited of 116 All 116 (JSON)