mlsanityData to 7 Oct 2026

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data

arXiv 2606.03094 ↗First posted 2 Jun 2026cs.LGcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Recent advances in language models have established reinforcement learning as the primary paradigm for eliciting self-correction and long-chain reasoning. While group relative policy optimization (GRPO) offers superior scalability by eliminating the critic network, deploying it on a central infrastructure entails collecting a large volume of data from distributed owners, which poses significant privacy risks. To address these concerns, we introduce federated GRPO (FGRPO), a framework designed to decentralize the fine-tuning of reasoning models across heterogeneous data owners. To effectively mitigate the instability caused by divergent reward scales across heterogeneous tasks, FGRPO incorporates an adaptive aggregation mechanism based on relative performance gain. By characterizing each client's improvement relative to its personalized historical baseline, the framework dynamically prioritizes effective learning trajectories regardless of local task difficulty. FGRPO ensures robust convergence on non-IID data while preserving data privacy.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 40 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

14 of 40 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Models10 Sep
  2. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback4 Aug
  3. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization18 May
  4. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models13 Apr
  5. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models28 Mar
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 7

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Qwen3-VL Technical Reportcited by 2,534
  5. 2024OpenAI o1 System Cardcited by 1,220
  6. 2025Group Sequence Policy Optimizationcited by 788
  7. 2025The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards
  2. 2026FedGRPO: Privately Optimizing Foundation Models with Group-Relative Rewards from Domain Client
  3. 2025CoRPO: Adding a Correctness Bias to GRPO Improves Generalization
  4. 2026iGRPO: Self-Feedback-Driven LLM Reasoning
  5. 2026Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
  6. 2026AMIR-GRPO: Inducing Implicit Preference Signals into GRPO
  7. 2026GRPO Training Dynamics for Small Language Models
  8. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?