# Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training

arXiv: https://arxiv.org/abs/2605.09608 · HTML: https://mlsanity.com/paper/2605.09608/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 70 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 20 of 20)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/2507.10485/) (2025)
- [Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning](https://mlsanity.com/paper/2509.25300/) (2025)
- [InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion](https://mlsanity.com/paper/2505.13893/) (2025)
- [Model Merging Scaling Laws in Large Language Models](https://mlsanity.com/paper/2509.24244/) (2025)
- [InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models](https://mlsanity.com/paper/2505.13878/) (2025)
- [Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity](https://mlsanity.com/paper/2509.21413/) (2025)
- [FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning](https://mlsanity.com/paper/2601.03938/) (2026)
- [MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging](https://mlsanity.com/paper/2602.13273/) (2026)
- [Scalable Strategies for Continual Learning with Replay](https://mlsanity.com/paper/2505.12512/) (2025)
- [GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay](https://mlsanity.com/paper/2508.04676/) (2025)
- [K-Merge: Online Continual Merging of Adapters for On-device Large Language Models](https://mlsanity.com/paper/2510.13537/) (2025)
- [Toward a Holistic Approach to Continual Model Merging](https://mlsanity.com/paper/2509.23592/) (2025)
- [InfiCoEvalChain: A Blockchain-Based Decentralized Framework for Collaborative LLM Evaluation](https://mlsanity.com/paper/2602.08229/) (2026)
- [From Coefficients to Directions: Rethinking Model Merging with Directional Alignment](https://mlsanity.com/paper/2512.00391/) (2025)

## Built on by (most cited first, 5 of 5)

- [Access Sets Matter: Budgeting Expert Reads for Scalable Weight-Space Model Merging](https://mlsanity.com/paper/2605.29489/) (2026)
- [From Parameters to Behaviors: A Survey of Model Fusion for Large Language Models](https://mlsanity.com/paper/2609.19553/) (2026)
- [Continual Learning in Transition](https://mlsanity.com/paper/2608.06216/) (2026)
- [Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs](https://mlsanity.com/paper/2609.32821/) (2026)
- [SMAT: Simple and Efficient Merge-Aware Training](https://mlsanity.com/paper/2609.33437/) (2026)

## Abstract (arXiv)

Continual post-training aims to extend large language models (LLMs) with new knowledge, skills, and behaviors, yet it remains unclear when sequential updates enable capability transfer and when they cause catastrophic forgetting. Existing methods mitigate forgetting through sequential fine-tuning, replay, regularization, or model merging, but offer limited criteria for determining when incorporating new updates is beneficial or harmful. In this work, we study LLM continual post-training through three questions: What drives forgetting? When do sequentially acquired capabilities transfer or interfere? How can compatibility be used to control update integration? We address these questions through task geometry: we represent each post-training task by its parameter update and study the covariance geometry induced by the update. Our central finding is that: forgetting can be considered as a state-relative update-integration failure, it arises when the covariance geometries induced by tasks misalign with the geometry of the evolving model state. Sequential updates transfer when they remain compatible with the model state shaped by previous updates, and interfere when state-relative geometry conflict becomes high. Motivated by this finding, we propose Geometry-Conflict Wasserstein Merging (GCWM), a data-free update-integration method that constructs a shared Wasserstein metric via Gaussian Wasserstein barycenters and uses geometry conflict to gate geometry-aware correction. Across Qwen3 0.6B--14B on domain-continual and capability-continual settings, GCWM consistently outperforms data-free baselines, improving retention and final performance without replay data. These results identify geometry conflict as both an explanatory signal for forgetting and a practical control signal for LLM continual post-training.