mlsanityData to 7 Oct 2026

On the Plasticity and Stability for Post-Training Large Language Models

arXiv 2602.06453 ↗First posted 6 Feb 2026cs.LGcited by 3

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads to destructive interference. Crucially, we argue that deterministic projection methods are suboptimal for GRPO as they overlook the intrinsic stochasticity of group-based gradient estimates. To address this, we propose Probabilistic Conflict Resolution (PCR), a Bayesian framework that models gradients as random variables. PCR dynamically arbitrates conflicts via an uncertainty-aware ``soft projection'' mechanism, optimizing the signal-to-noise ratio. Extensive experiments demonstrate that PCR significantly smooths the training trajectory and achieves superior performance in various reasoning tasks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 31 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

24 of 31 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients30 Jan
  2. 2025It Takes Two: Your GRPO Is Secretly DPO1 Oct
  3. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control5 Aug
  4. 2025BNPO: Beta Normalization Policy Optimization3 Jun
  5. 2025Stable Reinforcement Learning for Efficient Reasoning23 May
  6. 2025Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs15 May
  7. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning10 Mar

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2016Variational Inference: A Review for Statisticianscited by 1,466
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  10. 2025Group Sequence Policy Optimizationcited by 788
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  12. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
5 more
  1. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 72
  2. 2025BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clippingcited by 47
  3. 2025GVPO: Group Variance Policy Optimization for Large Language Model Post-Trainingcited by 32
  4. 2024Robust Multi-Task Learning with Excess Riskscited by 23
  5. 2025Group Causal Policy Optimization for Post-Training Large Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 5
  2. 2026Pareto-Guided Optimal Transport for Multi-Reward Alignmentcited by 0
  3. 2026RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models
  2. 2026GRPO Training Dynamics for Small Language Models
  3. 2025DaGRPO: Rectifying Gradient Conflict in Reasoning via Distinctiveness-Aware Group Relative Policy Optimization
  4. 2025M-GRPO: Stabilizing Self-Supervised Reinforcement Learning for Large Language Models with Momentum-Anchored Policy Optimization
  5. 2025Mitigating Think-Answer Mismatch in LLM Reasoning Through Noise-Aware Advantage Reweighting
  6. 2026Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
  7. 2025ESPO: Entropy Importance Sampling Policy Optimization
  8. 2026Stabilizing Reinforcement Learning for Diffusion Language Models