# On the Plasticity and Stability for Post-Training Large Language Models

arXiv: https://arxiv.org/abs/2602.06453 · HTML: https://mlsanity.com/paper/2602.06453/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 24 of 24)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.07572/) (2025)
- [MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance](https://mlsanity.com/paper/2405.17730/) (2024)
- [BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping](https://mlsanity.com/paper/2510.18927/) (2025)
- [GVPO: Group Variance Policy Optimization for Large Language Model Post-Training](https://mlsanity.com/paper/2504.19599/) (2025)
- [BNPO: Beta Normalization Policy Optimization](https://mlsanity.com/paper/2506.02864/) (2025)
- [Stable Reinforcement Learning for Efficient Reasoning](https://mlsanity.com/paper/2505.18086/) (2025)
- [It Takes Two: Your GRPO Is Secretly DPO](https://mlsanity.com/paper/2510.00977/) (2025)
- [Robust Multi-Task Learning with Excess Risks](https://mlsanity.com/paper/2402.02009/) (2024)
- [GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control](https://mlsanity.com/paper/2508.03772/) (2025)
- [Learning to Think: Information-Theoretic Reinforcement Fine-Tuning for LLMs](https://mlsanity.com/paper/2505.10425/) (2025)
- [Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients](https://mlsanity.com/paper/2601.23135/) (2026)
- [Group Causal Policy Optimization for Post-Training Large Language Models](https://mlsanity.com/paper/2508.05428/) (2025)

## Built on by (most cited first, 4 of 4)

- [Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization](https://mlsanity.com/paper/2605.21801/) (2026)
- [RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agents](https://mlsanity.com/paper/2606.20638/) (2026)
- [Pareto-Guided Optimal Transport for Multi-Reward Alignment](https://mlsanity.com/paper/2605.13155/) (2026)
- [GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models](https://mlsanity.com/paper/2608.17411/) (2026)

## Abstract (arXiv)

Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads to destructive interference. Crucially, we argue that deterministic projection methods are suboptimal for GRPO as they overlook the intrinsic stochasticity of group-based gradient estimates. To address this, we propose Probabilistic Conflict Resolution (PCR), a Bayesian framework that models gradients as random variables. PCR dynamically arbitrates conflicts via an uncertainty-aware ``soft projection'' mechanism, optimizing the signal-to-noise ratio. Extensive experiments demonstrate that PCR significantly smooths the training trajectory and achieves superior performance in various reasoning tasks.