mlsanityData to 14 Oct 2026

Harnessing the Power of Gradient-Based Simulations for Multi-Objective Optimization in Particle Accelerators

arXiv 2411.04817 ↗First posted 7 Nov 2024physics.acc-ph cs.NEcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Particle accelerator operation requires simultaneous optimization of multiple objectives. Multi-Objective Optimization (MOO) is particularly challenging due to trade-offs between the objectives. Evolutionary algorithms, such as genetic algorithm (GA), have been leveraged for many optimization problems, however, they do not apply to complex control problems by design. This paper demonstrates the power of differentiability for solving MOO problems using a Deep Differentiable Reinforcement Learning (DDRL) algorithm in particle accelerators. We compare DDRL algorithm with Model Free Reinforcement Learning (MFRL), GA and Bayesian Optimization (BO) for simultaneous optimization of heat load and trip rates in the Continuous Electron Beam Accelerator Facility (CEBAF). The underlying problem enforces strict constraints on both individual states and actions as well as cumulative (global) constraint for energy requirements of the beam. A physics-based surrogate model based on real data is developed. This surrogate model is differentiable and allows back-propagation of gradients. The results are evaluated in the form of a Pareto-front for two objectives. We show that the DDRL outperforms MFRL, BO, and GA on high dimensional problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 23 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

6 of 23 references.

References · 6

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  3. 2020Bayesian Neural Networks: An Introduction and Surveycited by 131
  4. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 118
  5. 2020Differential Machine Learningcited by 10
  6. 2021Developing Robust Digital Twins and Reinforcement Learning for Accelerator Control Systems at the Fermilab Boostercited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimisation of the Accelerator Control by Reinforcement Learning: A Simulation-Based Approach
  2. 2025Reinforcement Learning for Accelerator Beamline Control: a simulation-based approach
  3. 2024Cheetah: Bridging the Gap Between Machine Learning and Particle Accelerator Physics with High-Speed, Differentiable Simulations
  4. 2026RL-ABC: Reinforcement Learning for Accelerator Beamline Control
  5. 2026Adaptive Model-Based Reinforcement Learning for Orbit Feedback Control in NSLS-II Storage Ring
  6. 2024Large Language Models for Human-Machine Collaborative Particle Accelerator Tuning through Natural Language
  7. 2024Reactor Optimization Benchmark by Reinforcement Learning
  8. 2024Physics Instrument Design with Reinforcement Learning