# Game-Theoretic Reinforcement Learning based Topology-Agnostic Defense for Virtual Power Plants

arXiv: https://arxiv.org/abs/2610.15183 · HTML: https://mlsanity.com/paper/2610.15183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review](https://mlsanity.com/paper/2308.15736/) (2023)
- [Man-in-The-Middle Attacks and Defense in a Power System Cyber-Physical Testbed](https://mlsanity.com/paper/2102.11455/) (2021)
- [Deep Reinforcement Learning for DER Cyber-Attack Mitigation](https://mlsanity.com/paper/2009.13088/) (2020)
- [Inverter Probing for Power Distribution Network Topology Processing](https://mlsanity.com/paper/1802.06027/) (2018)
- [Moving-Target Defense Against Cyber-Physical Attacks in Power Grids via Game Theory](https://mlsanity.com/paper/2006.07697/) (2020)
- [A Robust Stackelberg Game for Cyber-Security Investment in Networked Control Systems](https://mlsanity.com/paper/2103.16611/) (2021)
- [Advancing Cyber-Attack Detection in Power Systems: A Comparative Study of Machine Learning and Graph Neural Network Approaches](https://mlsanity.com/paper/2411.02248/) (2024)
- [Detecting Generalized Replay Attacks via Time-Varying Dynamic Watermarking](https://mlsanity.com/paper/1909.08111/) (2019)
- [Probabilistic Voltage Sensitivity Analysis (PVSA) to Quantify Impact of High PV Penetration on Unbalanced Distribution System](https://mlsanity.com/paper/2009.05734/) (2020)
- [Physical Watermarking for Replay Attack Detection in Continuous-time Systems](https://mlsanity.com/paper/2103.00790/) (2021)
- [Defense against Joint Poison and Evasion Attacks: A Case Study of DERMS](https://mlsanity.com/paper/2405.02989/) (2024)

## Abstract (arXiv)

The rapid proliferation of distributed energy resources (DERs) aggregated by virtual power plants (VPPs) elevates cybersecurity risk by expanding the attack surface. A coordinated cyberattack has the potential to disrupt distribution-system, bulk-system, and market operations. Existing cyberphysical defense strategies largely assume access to grid network models and operating conditions, which may not be available to third-party aggregators. This paper proposes a novel hybrid, perturbation-based cyber-physical defense for VPP-operated DERs that blends game-theoretic planning with model-free reinforcement learning (RL) under a Bayesian belief-update framework. Topology agnosticism is defined from the VPP information and implementation perspective: the VPP does not require access to the feeder topology, line parameters, switching configuration, or full system state. This work advances the state of the art through three contributions: (1) a multi-output Gaussian process with Hamiltonian Monte Carlo for discrete attacker-type inference, probability of detection, and confidence of detection for modeling the belief distribution; (2) moment-matched Gaussian propagation of posterior beliefs across episodes; and (3) an RL policy that adaptively integrates Stackelberg leader (VPP) actions with exploration. Furthermore, we establish equilibrium existence and derive an episode-level closed-loop utility bound connecting attacker-belief uncertainty with GT-RL action alignment. Evaluations on IEEE 13- and 123-Bus feeders with DER portfolios demonstrate effective detection under attacker-model uncertainty and evaluated feeder re-configuration conditions, while requiring no explicit feeder-topology knowledge. Overall, the results indicate that combining bi-level game-theoretic planning, nonparametric Bayesian inference, and adaptive RL is a promising direction for VPP cybersecurity.