mlsanityData to 14 Oct 2026

Offline Reinforcement Learning for Rotation Profile Control in Tokamaks

arXiv 2605.05857 ↗First posted 7 May 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Tokamaks remain leading candidates for achieving practical fusion energy, yet many important control problems inside these devices are still difficult or unsolved. One such challenge is controlling the plasma rotation profile, which strongly influences stability, confinement, and transport. While the average rotation can be controlled, controlling the full profile is challenging due to high dimensionality, response to multiple actuators and dependence on plasma condition. Learning-based control methods, such as reinforcement learning (RL), provide a potential solution to this challenging problem with ability to model complex interactions leading to effective multi-input multi-output control. However, learning such policies is challenging due to the lack of accurate simulators that can model the rotation profile dynamics. In this work, we investigate the use of offline RL and offline model-based RL algorithms for rotation profile control, training them solely on historical data from the DIII-D tokamak. Our final method uses probabilistic models of plasma dynamics to generate rollouts for RL training. We deploy this policy on the DIII-D Tokamak and observe promising real-world results. We conclude by highlighting key challenges and insights from training and deploying an RL policy on a complex physical device while using only limited past data.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 27 of its 57 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

27 of 57 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  10. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  11. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
15 more
  1. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  2. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  3. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  4. 2019Goal-conditioned Imitation Learningcited by 162
  5. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  6. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  7. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  8. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systemscited by 11
  9. 2024Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learningcited by 11
  10. 2023Towards practical reinforcement learning for tokamak magnetic controlcited by 9
  11. 2024Full Shot Predictions for the DIII-D Tokamak via Deep Recurrent Networkscited by 9
  12. 2022Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarioscited by 8
  13. 2025Learning Plasma Dynamics and Robust Rampdown Trajectories with Predict-First Experiments at TCVcited by 4
  14. 2025Multi-Timescale Dynamics Model Bayesian Optimization for Plasma Stabilization in Tokamakscited by 3
  15. 2025Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Offline Reinforcement Learning for Plasma Control in Nuclear Fusion: Codebase and Benchmarkcited by 0
  2. 2026Real-Time Plasma State Prediction via FPGA-Accelerated Quantized Recurrent Probabilistic Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research
  2. 2024Active Disruption Avoidance and Trajectory Design for Tokamak Ramp-downs with Neural Differential Equations and Reinforcement Learning
  3. 2025Plasma Shape Control via Zero-shot Generative Reinforcement Learning
  4. 2026Dynamic Plasma Shape Control with Arbitrary Sensor Subsets ↗
  5. 2025Enabling Integrated AI Control on DIII-D: A Control System Design with State-of-the-art Experiments
  6. 2026Advantage-level Aggregation Reinforcement Learning for X-point Target Magnetic Configuration Control in an EXL-50U Experiment-Calibrated Simulation Environment
  7. 2024Multistep Criticality Search and Power Shaping in Microreactors with Reinforcement Learning
  8. 2026IGNITE Tokamak World Model Architecture