mlsanityData to 14 Oct 2026

Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity

arXiv 2605.27385 ↗First posted 10 Apr 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Federated reinforcement learning (FedRL) enables multiple agents to collaboratively train a global policy without sharing raw data, making it ideal for privacy-sensitive applications. However, FedRL faces challenges in heterogeneous environments where differing state-transition dynamics lead to non-identical input distributions and imbalanced parameter updates during aggregation. Therefore, this paper develops a personalized observation normalization (PON) method, allowing each agent to locally normalize raw state inputs using a continuously updated running mean and variance. This design ensures consistent scaling of local feature without overshadowing across agents during aggregation. Furthermore, we demonstrate that sharing normalization parameters across agents is ineffective due to the diverse local input distributions, which highlights the necessity of personalized statistics. Experiments on heterogeneous MuJoCo tasks show that our developed PON accelerates training and achieves superior performance compared to baseline methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 34 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

15 of 34 references.

References · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  6. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  7. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  8. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  9. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  10. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
  11. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  12. 2022Federated Reinforcement Learning with Environment Heterogeneitycited by 63
3 more
  1. 2022CalFAT: Calibrated Federated Adversarial Training with Label Skewnesscited by 20
  2. 2022Rethinking Normalization Methods in Federated Learningcited by 19
  3. 2023DeliverAI: Reinforcement Learning Based Distributed Path-Sharing Network for Food Deliveriescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representations
  2. 2025FedHPD: Heterogeneous Federated Reinforcement Learning via Policy Distillation
  3. 2025Federated Reinforcement Learning in Heterogeneous Environments
  4. 2026Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients
  5. 2026Exploration-Driven Personalized Federated Reinforcement Learning via Intrinsic Motivation
  6. 2025Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis
  7. 2026FedGuide: Diffusion Prior Alignment and Value Baseline Guidance for Heterogeneous Federated Reinforcement Learning
  8. 2025Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning