# Active Disruption Avoidance and Trajectory Design for Tokamak Ramp-downs with Neural Differential Equations and Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.09387 · HTML: https://mlsanity.com/paper/2402.09387/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Reinforcement and Imitation Learning for Diverse Visuomotor Skills](https://mlsanity.com/paper/1802.09564/) (2018)
- [Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learning](https://mlsanity.com/paper/2305.14154/) (2023)
- [Model Predictive Contouring Control for Time-Optimal Quadrotor Flight](https://mlsanity.com/paper/2108.13205/) (2021)

## Built on by (most cited first, 3 of 3)

- [Solving Minimum-Cost Reach Avoid using Reinforcement Learning](https://mlsanity.com/paper/2410.22600/) (2024)
- [Learning Plasma Dynamics and Robust Rampdown Trajectories with Predict-First Experiments at TCV](https://mlsanity.com/paper/2502.12327/) (2025)
- [TokaMark: A Comprehensive Benchmark for MAST Tokamak Plasma Models](https://mlsanity.com/paper/2602.10132/) (2026)

## Abstract (arXiv)

The tokamak offers a promising path to fusion energy, but plasma disruptions pose a major economic risk, motivating considerable advances in disruption avoidance. This work develops a reinforcement learning approach to this problem by training a policy to safely ramp-down the plasma current while avoiding limits on a number of quantities correlated with disruptions. The policy training environment is a hybrid physics and machine learning model trained on simulations of the SPARC primary reference discharge (PRD) ramp-down, an upcoming burning plasma scenario which we use as a testbed. To address physics uncertainty and model inaccuracies, the simulation environment is massively parallelized on GPU with randomized physics parameters during policy training. The trained policy is then successfully transferred to a higher fidelity simulator where it successfully ramps down the plasma while avoiding user-specified disruptive limits. We also address the crucial issue of safety criticality by demonstrating that a constraint-conditioned policy can be used as a trajectory design assistant to design a library of feed-forward trajectories to handle different physics conditions and user settings. As a library of trajectories is more interpretable and verifiable offline, we argue such an approach is a promising path for leveraging the capabilities of reinforcement learning in the safety-critical context of burning plasma tokamaks. Finally, we demonstrate how the training environment can be a useful platform for other feed-forward optimization approaches by using an evolutionary algorithm to perform optimization of feed-forward trajectories that are robust to physics uncertainty