# Offline Reinforcement Learning for Distribution-Grid Protection

arXiv: https://arxiv.org/abs/2609.24703 · HTML: https://mlsanity.com/paper/2609.24703/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 7 of 7)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [A Scoping Review of Machine Learning Applications in Power System Protection and Disturbance Management](https://mlsanity.com/paper/2509.09053/) (2025)
- [Robustness Evaluation of Machine Learning Models for Fault Classification and Localization In Power System Protection](https://mlsanity.com/paper/2512.15385/) (2025)
- [Nested Reinforcement Learning Based Control for Protective Relays in Power Distribution Systems](https://mlsanity.com/paper/1906.10815/) (2019)
- [A Systematic Evaluation of Machine Learning Methods for Fault Detection and Line Identification in Electrical Power Grids](https://mlsanity.com/paper/2609.16744/) (2026)
- [A simulation based dataset of faults and events for machine learning in power systems](https://mlsanity.com/paper/2608.19777/) (2026)

## Abstract (arXiv)

Data-driven protection may complement conventional relays in distribution grids whose operating conditions vary with distributed generation, switching events, and changing short-circuit levels. We study line-selective tripping from static trajectories of a realistically simulated CIGRE medium-voltage network using offline reinforcement learning. A convolutional Q-network receives causal voltage-current phasor and apparent-impedance features, optionally together with raw waveforms, and is trained with conservative Q-learning (CQL). A controlled sensitivity study evaluates two observation windows, reward variants, and three CQL weights under a common split and training protocol; one exploratory post-hoc run additionally increases the discount factor from γ=0.95 to 0.99. On 225 held-out episodes, the best per-timestep result is obtained with combined input and CQL weight α=0.9, reaching precision 0.9993, recall 0.9496, and F1-score 0.9738. Because dense per-timestep scores do not encode the terminal semantics of relay operation, we also evaluate the first non-wait action in each episode. The default combined-input agent selects the correct line-trip action first in 98.13% of 214 fault episodes, but trips in 72.73% of the 11 non-fault episodes. In the post-hoc run, the corresponding rates are 98.60% and 54.55%, respectively. The results show that dense predictive performance and terminal protection behavior can lead to different model rankings. Offline CQL therefore demonstrates strong faulted-line selection on the simulated fault episodes, while the static trajectories, small non-fault set, and single-seed post-hoc design preclude conclusions about practical relay security or deployment readiness.