# Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination

arXiv: https://arxiv.org/abs/2607.12763 · HTML: https://mlsanity.com/paper/2607.12763/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 14 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [A Review of Federated Learning in Energy Systems](https://mlsanity.com/paper/2208.10941/) (2022)
- [A Federated DRL Approach for Smart Micro-Grid Energy Control with Distributed Energy Resources](https://mlsanity.com/paper/2211.03430/) (2022)
- [FedHQL: Federated Heterogeneous Q-Learning](https://mlsanity.com/paper/2301.11135/) (2023)
- [A review of federated learning in renewable energy applications: Potential, challenges, and future directions](https://mlsanity.com/paper/2312.11220/) (2023)
- [Federated Reinforcement Learning with Constraint Heterogeneity](https://mlsanity.com/paper/2405.03236/) (2024)

## Abstract (arXiv)

Federated Reinforcement Learning (FedRL) enables coordination of distributed energy resources without sharing raw local data, but standard aggregation methods such as FedAvg do not account for system-level constraints, often leading to unsafe global behavior. In this work, we study constraint-aware aggregation for federated reinforcement learning in distributed energy coordination. We propose aggregation rules that incorporate both local performance and estimated constraint violation into the server-side update. Among these, a simple penalty-based rule, wi propto Ri - α Vi, consistently provides the most reliable trade-off between reward and safety, without requiring dual optimization or modifications to local training. textcolorblackWe evaluate our approach on DairyGridEnv, a benchmark modeling multiple farms coordinating battery storage under stochastic demand and a shared grid capacity constraint, and further assess robustness using real load-driven demand profiles from Finland and the German FIELD dataset. Across multiple seeds, penalty-based aggregation substantially reduces violations while improving reward relative to FedAvg in both synthetic and real load-driven settings. A combined reward-violation scheme exposes a tunable trade-off via λ, but is less stable. These results demonstrate that lightweight aggregation strategies can substantially improve empirical safety in federated reinforcement learning while preserving standard communication protocols.