mlsanityData to 7 Oct 2026

Gradients are Not All You Need

arXiv 2111.05803 ↗First posted 10 Nov 2021cs.LG stat.MLcited by 82

Not on the map. Why?

Abstract (arXiv)

Differentiable programming techniques are widely used in the community and are responsible for the machine learning renaissance of the past several decades. While these methods are powerful, they have limits. In this short report, we discuss a common chaos based failure mode which appears in a variety of differentiable circumstances, ranging from recurrent neural networks and numerical physics simulation to training learned optimizers. We trace this failure to the spectrum of the Jacobian of the system under study, and provide criteria for when a practitioner might expect this failure to spoil their differentiation based optimization algorithms.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 44 of its 110 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

44 of 110 references.

References · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  3. 2018World Modelscited by 1,444
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  6. 2016Learning to learn by gradient descent by gradient descentcited by 616
  7. 2019Deep Equilibrium Modelscited by 603
  8. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  9. 2017Convolutional Sequence to Sequence Learningcited by 550
  10. 2019Meta-Learning with Implicit Gradientscited by 523
  11. 2021Machine learning accelerated computational fluid dynamicscited by 461
  12. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
32 more
  1. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  2. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  3. 2016Deep Learning without Poor Local Minimacited by 233
  4. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  5. 2017Regularizing and Optimizing LSTM Language Modelscited by 205
  6. 2021Efficient and Modular Implicit Differentiationcited by 199
  7. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  8. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  9. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  10. 2016Accelerating Eulerian Fluid Simulation With Convolutional Networkscited by 128
  11. 2020Differentiable Rendering: A Surveycited by 123
  12. 2015Gradient Estimation Using Stochastic Computation Graphscited by 120
  13. 2015Neural GPUs Learn Algorithmscited by 118
  14. 2017Learned Optimizers that Scale and Generalizecited by 109
  15. 2016A Differentiable Physics Engine for Deep Learning in Roboticscited by 107
  16. 2019On the Impact of the Activation Function on Deep Neural Networks Trainingcited by 101
  17. 2021PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physicscited by 100
  18. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  19. 2017Mean Field Residual Networks: On the Edge of Chaoscited by 83
  20. 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 77
  21. 2018Understanding Short-Horizon Bias in Stochastic Meta-Optimizationcited by 68
  22. 2020Model-Augmented Actor-Critic: Backpropagating through Pathscited by 61
  23. 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
  24. 2017Learning Gradient Descent: Better Generalization and Longer Horizonscited by 50
  25. 2021Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contactcited by 50
  26. 2021Vector Quantized Models for Planningcited by 47
  27. 2017Unbiasing Truncated Backpropagation Through Timecited by 37
  28. 2016Capacity and Trainability in Recurrent Neural Networkscited by 26
  29. 2020Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Functioncited by 26
  30. 2020Super-efficiency of automatic differentiation for functions defined as a minimumcited by 24
  31. 2021Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarkingcited by 10
  32. 2016Sensitivity analysis on chaotic dynamical system by Non-Intrusive Least Square Shadowing (NILSS)cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 23; 2023: 22; 2024: 17; 2025: 9; 2026: 10.

82 papers.

Sort:

By area:

  1. 2022Forecasting Global Weather with Graph Neural Networkscited by 165
  2. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  3. 2022Flow Annealed Importance Sampling Bootstrapcited by 124
  4. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 109
  5. 2022Discovered Policy Optimisationcited by 103
  6. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 85
  7. 2022Tutorial on amortized optimizationcited by 83
  8. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 83
  9. 2024ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEscited by 78
  10. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  11. 2022evosax: JAX-based Evolution Strategiescited by 39
  12. 2022Differentiable Dynamics for Articulated 3d Human Motion Reconstructioncited by 38
38 more
  1. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  2. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 22
  3. 2022Differentiable Programming for Earth System Modelingcited by 21
  4. 2024Discovering Temporally-Aware Reinforcement Learning Algorithmscited by 18
  5. 2021Input Convex Gradient Networkscited by 17
  6. 2022Latent Neural ODEs with Sparse Bayesian Multiple Shootingcited by 17
  7. 2024PWM: Policy Learning with Multi-Task World Modelscited by 17
  8. 2022Training Efficient Controllers via Analytic Policy Gradientcited by 16
  9. 2022Adversarial Cheap Talkcited by 16
  10. 2022Transformer-Based Learned Optimizationcited by 13
  11. 2024chemtrain: Learning Deep Potential Models via Automatic Differentiation and Statistical Physicscited by 12
  12. 2022Plateau-reduced Differentiable Path Tracingcited by 10
  13. 2024Do Transformer World Models Give Better Policy Gradients?cited by 10
  14. 2023Fractal Landscapes in Policy Optimizationcited by 8
  15. 2023Gradient Informed Proximal Policy Optimizationcited by 8
  16. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 8
  17. 2023Differentiable Simulations for Enhanced Sampling of Rare Eventscited by 7
  18. 2023Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Singlecited by 7
  19. 2023Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithmscited by 7
  20. 2023Differentiable Visual Computing for Inverse Problems and Machine Learningcited by 7
  21. 2022Neural Payoff Machines: Predicting Fair and Stable Payoff Allocations Among Team Memberscited by 6
  22. 2025Relative Entropy Pathwise Policy Optimizationcited by 6
  23. 2023Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategiescited by 5
  24. 2023Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphicscited by 5
  25. 2024Behaviour Distillationcited by 5
  26. 2025How Should We Meta-Learn Reinforcement Learning Algorithms?cited by 5
  27. 2025RoboSSM: Scalable In-context Imitation Learning via State-Space Modelscited by 5
  28. 2022Leveraging Reward Gradients For Reinforcement Learning in Differentiable Physics Simulationscited by 4
  29. 2022Learning Pair Potentials using Differentiable Simulationscited by 4
  30. 2023Arbitrary Order Meta-Learning with Simple Population-Based Evolutioncited by 4
  31. 2023NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applicationscited by 4
  32. 2024Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial Statescited by 4
  33. 2025Accelerating Visual-Policy Learning through Parallel Differentiable Simulationcited by 4
  34. 2022A survey of deep learning optimizers -- first and second order methodscited by 3
  35. 2023Inductive biases in deep learning models for weather predictioncited by 3
  36. 2023Learning Space-Time Continuous Neural PDEs from Partially Observed Statescited by 3
  37. 2024Stabilizing Backpropagation Through Time to Learn Complex Physicscited by 3
  38. 2024Narrowing the Focus: Learned Optimizers for Pretrained Modelscited by 3

50 most cited of 82 All 82 (JSON)