# Stabilizing Backpropagation Through Time to Learn Complex Physics

arXiv: https://arxiv.org/abs/2405.02041 · HTML: https://mlsanity.com/paper/2405.02041/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 30 of 33)

- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Deep Fluids: A Generative Network for Parameterized Fluid Simulations](https://mlsanity.com/paper/1806.02071/) (2018)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Do Differentiable Simulators Give Better Policy Gradients?](https://mlsanity.com/paper/2202.00817/) (2022)
- [Gradients are Not All You Need](https://mlsanity.com/paper/2111.05803/) (2021)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Deep Dynamical Modeling and Control of Unsteady Fluid Flows](https://mlsanity.com/paper/1805.07472/) (2018)
- [Physics-based Deep Learning](https://mlsanity.com/paper/2109.05237/) (2021)
- [SPNets: Differentiable Fluid Dynamics for Deep Neural Networks](https://mlsanity.com/paper/1806.06094/) (2018)
- [PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos](https://mlsanity.com/paper/1902.01240/) (2019)
- [Structured agents for physical construction](https://mlsanity.com/paper/1904.03177/) (2019)
- [Efficient Differentiable Simulation of Articulated Bodies](https://mlsanity.com/paper/2109.07719/) (2021)
- [Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons](https://mlsanity.com/paper/2202.06988/) (2022)
- [Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics](https://mlsanity.com/paper/2210.06036/) (2022)
- [Differentiable Molecular Simulations for Control and Learning](https://mlsanity.com/paper/2003.00868/) (2020)

## Built on by (most cited first, 3 of 3)

- [A Neural Material Point Method for Particle-based Emulation](https://mlsanity.com/paper/2408.15753/) (2024)
- [ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations](https://mlsanity.com/paper/2406.04709/) (2024)
- [Integrating Mechanistic and Data-Driven Models for Neurological Disorders through Differentiable Programming](https://mlsanity.com/paper/2606.06094/) (2026)

## Abstract (arXiv)

Of all the vector fields surrounding the minima of recurrent learning setups, the gradient field with its exploding and vanishing updates appears a poor choice for optimization, offering little beyond efficient computability. We seek to improve this suboptimal practice in the context of physics simulations, where backpropagating feedback through many unrolled time steps is considered crucial to acquiring temporally coherent behavior. The alternative vector field we propose follows from two principles: physics simulators, unlike neural networks, have a balanced gradient flow, and certain modifications to the backpropagation pass leave the positions of the original minima unchanged. As any modification of backpropagation decouples forward and backward pass, the rotation-free character of the gradient field is lost. Therefore, we discuss the negative implications of using such a rotational vector field for optimization and how to counteract them. Our final procedure is easily implementable via a sequence of gradient stopping and component-wise comparison operations, which do not negatively affect scalability. Our experiments on three control problems show that especially as we increase the complexity of each task, the unbalanced updates from the gradient can no longer provide the precise control signals necessary while our method still solves the tasks. Our code can be found at https://github.com/tum-pbs/StableBPTT.