mlsanityData to 7 Oct 2026

Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

arXiv 2312.17372 ↗First posted 28 Dec 2023cs.LG cs.AI physics.acc-phcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 27 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

14 of 27 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  8. 2020Hopfield Networks is All You Needcited by 461
  9. 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 154
  10. 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 139
  11. 2023On Sparse Modern Hopfield Modelcited by 41
  12. 2022History Compression via Language Models in Reinforcement Learningcited by 39
2 more
  1. 2023ML-based Real-Time Control at the Edge: An Approach Using hls4mlcited by 3
  2. 2015A New PID Neural Network Controller Design for Nonlinear Processescited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Human-in-the-loop Reinforcement Learning for Data Quality Monitoring in Particle Physics Experimentscited by 2