mlsanityData to 7 Oct 2026

Bridging Discrete and Backpropagation: Straight-Through and Beyond

arXiv 2304.08612 ↗First posted 17 Apr 2023cs.LG cs.AI cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Backpropagation, the cornerstone of deep learning, is limited to computing gradients for continuous variables. This limitation poses challenges for problems involving discrete latent variables. To address this issue, we propose a novel approach to approximate the gradient of parameters involved in generating discrete latent variables. First, we examine the widely used Straight-Through (ST) heuristic and demonstrate that it works as a first-order approximation of the gradient. Guided by our findings, we propose ReinMax, which achieves second-order accuracy by integrating Heun's method, a second-order numerical method for solving ODEs. ReinMax does not require Hessian or other second-order derivatives, thus having negligible computation overheads. Extensive experimental results on various tasks demonstrate the superiority of ReinMax over the state of the art. Implementations are released at https://github.com/microsoft/ReinMax.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

25 of 41 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  7. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  8. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  9. 2016Self-critical Sequence Training for Image Captioningcited by 544
  10. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  11. 2014A* Samplingcited by 150
  12. 2018ListOps: A Diagnostic Dataset for Latent Tree Learningcited by 128
13 more
  1. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
  2. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  3. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  4. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 84
  5. 2017REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelscited by 78
  6. 2013Deep AutoRegressive Networkscited by 60
  7. 2017Learning to Compose Task-Specific Tree Structurescited by 48
  8. 2020Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimatorcited by 37
  9. 2015MuProp: Unbiased Backpropagation for Stochastic Neural Networkscited by 36
  10. 2020DisARM: An Antithetic Gradient Estimator for Binary Latent Variablescited by 27
  11. 2022Gradient Estimation with Discrete Stein Operatorscited by 22
  12. 2021Coupled Gradient Estimators for Discrete Latent Variablescited by 9
  13. 2022Training Discrete Deep Generative Models via Gapped Straight-Through Estimatorcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 10; 2026: 12.

31 papers.

Sort:

By area:

  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  2. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  3. 2024DISP-LLM: Dimension-Independent Structural Pruning for Large Language Modelscited by 24
  4. 2025Adaptive Layer-skipping in Pre-trained LLMscited by 24
  5. 2025MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Controlcited by 20
  6. 2023Sparse Backpropagation for MoE Trainingcited by 16
  7. 2024GRIN: GRadient-INformed MoEcited by 16
  8. 2024FedBAT: Communication-Efficient Federated Learning via Learnable Binarizationcited by 9
  9. 2025Quantum-PEFT: Ultra parameter-efficient fine-tuningcited by 9
  10. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 5
  11. 2025Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projectioncited by 4
  12. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
19 more
  1. 2025CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Trainingcited by 4
  2. 2026Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusioncited by 3
  3. 2024VQSynery: Robust Drug Synergy Prediction With Vector Quantization Mechanismcited by 2
  4. 2026Model Compression with Exact Budget Constraints via Riemannian Manifoldscited by 2
  5. 2024Revisiting Score Function Estimators for k-Subset Samplingcited by 1
  6. 2025An Introduction to Discrete Variational Autoencoderscited by 1
  7. 2026Neural QAOA2: Differentiable Joint Graph Partitioning and Parameter Initialization for Quantum Combinatorial Optimizationcited by 1
  8. 2024Multi-objective Differentiable Neural Architecture Searchcited by 0
  9. 2025Variational Adaptive Noise and Dropout towards Stable Recurrent Neural Networkscited by 0
  10. 2025Feature Space Analysis by Guided Diffusion Modelcited by 0
  11. 2026Categorical Reparameterization with Denoising Diffusion modelscited by 0
  12. 2026Beyond ReinMax: Low-Variance Gradient Estimators for Discrete Latent Variablescited by 0
  13. 2026GO-GenZip: Goal-Oriented Generative Sampling and Hybrid Compressioncited by 0
  14. 2026PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Promptscited by 0
  15. 2026INSHAPE: Instance-Level Shapelets for Interpretable Time-Series Classificationcited by 0
  16. 2026LLM Compression with Jointly Optimizing Architectural and Quantization choicescited by 0
  17. 2026Neural Network Quantization by Learning Low-Loss Subspacescited by 0
  18. 2026End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inferencecited by 0
  19. 2026Dual-Primal Graph VAEs for Noisy Label Aggregationcited by 0