# Human-in-the-loop Reinforcement Learning for Data Quality Monitoring in Particle Physics Experiments

arXiv: https://arxiv.org/abs/2405.15508 · HTML: https://mlsanity.com/paper/2405.15508/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [Detector monitoring with artificial neural networks at the CMS experiment at the CERN Large Hadron Collider](https://mlsanity.com/paper/1808.00911/) (2018)
- [Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning for Particle Accelerator Control Demonstrated on the FERMI FEL](https://mlsanity.com/paper/2012.09737/) (2020)
- [Autonomous Control of a Particle Accelerator using Deep Reinforcement Learning](https://mlsanity.com/paper/2010.08141/) (2020)
- [Developing Robust Digital Twins and Reinforcement Learning for Accelerator Control Systems at the Fermilab Booster](https://mlsanity.com/paper/2105.12847/) (2021)
- [Spatio-Temporal Anomaly Detection with Graph Networks for Data Quality Monitoring of the Hadron Calorimeter](https://mlsanity.com/paper/2311.04190/) (2023)
- [Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e](https://mlsanity.com/paper/2312.17372/) (2023)

## Built on by (most cited first, 2 of 2)

- [Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection](https://mlsanity.com/paper/2408.16612/) (2024)
- [DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments](https://mlsanity.com/paper/2501.19237/) (2025)

## Abstract (arXiv)

Data Quality Monitoring (DQM) is a crucial task in large particle physics experiments, since detector malfunctioning can compromise the data. DQM is currently performed by human shifters, which is costly and results in limited accuracy. In this work, we provide a proof-of-concept for applying human-in-the-loop Reinforcement Learning (RL) to automate the DQM process while adapting to operating conditions that change over time. We implement a prototype based on the Proximal Policy Optimization (PPO) algorithm and validate it on a simplified synthetic dataset. We demonstrate how a multi-agent system can be trained for continuous automated monitoring during data collection, with human intervention actively requested only when relevant. We show that random, unbiased noise in human classification can be reduced, leading to an improved accuracy over the baseline. Additionally, we propose data augmentation techniques to deal with scarce data and to accelerate the learning process. Finally, we discuss further steps needed to implement the approach in the real world, including protocols for periodic control of the algorithm's outputs.