mlsanityData to 7 Oct 2026

Gradient Estimation Using Stochastic Computation Graphs

arXiv 1506.05254 ↗First posted 17 Jun 2015cs.LGcited by 120

Not on the map. Why?

Abstract (arXiv)

In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables, which might be part of a probabilistic model or the external world. Estimating the gradient of this loss function, using samples, lies at the core of gradient-based learning algorithms for these problems. We introduce the formalism of stochastic computation graphs---directed acyclic graphs that include both deterministic functions and conditional probability distributions---and describe how to easily and automatically derive an unbiased estimator of the loss function's gradient. The resulting algorithm for computing the gradient estimator is a simple modification of the standard backpropagation algorithm. The generic scheme we propose unifies estimators derived in variety of prior work, along with variance-reduction techniques therein. It could assist researchers in developing intricate models involving a combination of stochastic and deterministic operations, enabling, for example, attention, memory, and control actions.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 0; 2021: 22; 2022: 27; 2023: 20; 2024: 17; 2025: 7; 2026: 25.

120 papers.

Sort:

By area:

  1. 2021A Survey on Deep Semi-supervised Learningcited by 250
  2. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 103
  3. 2021Implicit MLE: Backpropagating Through Discrete Exponential Family Distributionscited by 97
  4. 2023Soft Merging of Experts with Adaptive Routingcited by 88
  5. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 83
  6. 2021Gradients are Not All You Needcited by 82
  7. 2022Efficient Transformers with Dynamic Token Poolingcited by 75
  8. 2022C2N: Practical Generative Noise Modeling for Real-World Denoisingcited by 56
  9. 2024ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routingcited by 53
  10. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  11. 2022A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inferencecited by 50
  12. 2022A Variational Perspective on Generative Flow Networkscited by 39
38 more
  1. 2022Automatic Differentiation of Programs with Discrete Randomnesscited by 28
  2. 2025On a few pitfalls in KL divergence gradient estimation for RLcited by 27
  3. 2021Generalizable Human Pose Triangulationcited by 24
  4. 2019Learning Belief Representations for Imitation Learning in POMDPscited by 23
  5. 2021Parallel Attention Network with Sequence Matching for Video Groundingcited by 22
  6. 2021CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signalscited by 22
  7. 2022Gradient Estimation with Discrete Stein Operatorscited by 22
  8. 2022Sparse Graph Learning from Spatiotemporal Time Seriescited by 22
  9. 2022General Intelligence Requires Rethinking Explorationcited by 20
  10. 2023CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Studycited by 19
  11. 2021Differentiable Particle Filtering without Modifying the Forward Passcited by 18
  12. 2024Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulationcited by 18
  13. 2021Emergent Communication under Competitioncited by 17
  14. 2024PWM: Policy Learning with Multi-Task World Modelscited by 17
  15. 2021Nested Variational Inferencecited by 14
  16. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 14
  17. 2021A unified view of likelihood ratio and reparameterization gradientscited by 12
  18. 2025Neurosymbolic Diffusion Modelscited by 12
  19. 2021Storchastic: A Framework for General Stochastic Automatic Differentiationcited by 11
  20. 2022Deep Structural Causal Shape Modelscited by 11
  21. 2023Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segmentscited by 11
  22. 2024SubgoalXL: Subgoal-based Expert Learning for Theorem Provingcited by 11
  23. 2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
  24. 2022Plateau-reduced Differentiable Path Tracingcited by 10
  25. 2023Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysiscited by 10
  26. 2021Characterizing the Gap Between Actor-Critic and Policy Gradientcited by 9
  27. 2022Consistent Dropout for Policy Gradient Reinforcement Learningcited by 9
  28. 2022Safe Neurosymbolic Learning with Differentiable Symbolic Executioncited by 9
  29. 2024ULLER: A Unified Language for Learning and Reasoningcited by 9
  30. 2024Stabilizing Policy Gradients for Stochastic Differential Equations via Consistency with Perturbation Processcited by 8
  31. 2021Snippet Policy Network for Multi-class Varied-length ECG Early Classificationcited by 7
  32. 2023Differentiable Simulations for Enhanced Sampling of Rare Eventscited by 7
  33. 2023Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Singlecited by 7
  34. 2021Learning Proposals for Probabilistic Programs with Inference Combinatorscited by 6
  35. 2021PriFit: Learning to Fit Primitives Improves Few Shot Point Cloud Segmentationcited by 6
  36. 2022Deep invariant networks with differentiable augmentation layerscited by 6
  37. 2023Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physicscited by 6
  38. 2024PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample Consensuscited by 6

50 most cited of 120 All 120 (JSON)