# Bayesian Reinforcement Learning for Automatic Voltage Control under Cyber-Induced Uncertainty

arXiv: https://arxiv.org/abs/2305.16469 · HTML: https://mlsanity.com/paper/2305.16469/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 13 of 13)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Faster Algorithms for High-Dimensional Robust Covariance Estimation](https://mlsanity.com/paper/1906.04661/) (2019)
- [Graph Neural Networks Based Detection of Stealth False Data Injection Attacks in Smart Grids](https://mlsanity.com/paper/2104.02012/) (2021)
- [Bayesian Reinforcement Learning in Factored POMDPs](https://mlsanity.com/paper/1811.05612/) (2018)
- [Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense](https://mlsanity.com/paper/2007.10457/) (2020)
- [Bayesian Policy Optimization for Model Uncertainty](https://mlsanity.com/paper/1810.01014/) (2018)
- [Adversarial Training for a Continuous Robustness Control Problem in Power Systems](https://mlsanity.com/paper/2012.11390/) (2020)
- [Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training](https://mlsanity.com/paper/2110.08956/) (2021)
- [Multi-Source Data Fusion for Cyberattack Detection in Power Systems](https://mlsanity.com/paper/2101.06897/) (2021)
- [Design and Evaluation of A Cyber-Physical Resilient Power System Testbed](https://mlsanity.com/paper/2011.13552/) (2020)

## Abstract (arXiv)

Voltage control is crucial to large-scale power system reliable operation, as timely reactive power support can help prevent widespread outages. However, there is currently no built in mechanism for power systems to ensure that the voltage control objective to maintain reliable operation will survive or sustain the uncertainty caused under adversary presence. Hence, this work introduces a Bayesian Reinforcement Learning (BRL) approach for power system control problems, with focus on sustained voltage control under uncertainty in a cyber-adversarial environment. This work proposes a data-driven BRL-based approach for automatic voltage control by formulating and solving a Partially-Observable Markov Decision Problem (POMDP), where the states are partially observable due to cyber intrusions. The techniques are evaluated on the WSCC and IEEE 14 bus systems. Additionally, BRL techniques assist in automatically finding a threshold for exploration and exploitation in various RL techniques.