# Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions

arXiv: https://arxiv.org/abs/2603.09938 · HTML: https://mlsanity.com/paper/2603.09938/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2026.

## Built on by (most cited first, 6 of 6)

- [Can Heterogeneous Language Models Be Fused?](https://mlsanity.com/paper/2604.01674/) (2026)
- [Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization](https://mlsanity.com/paper/2605.24960/) (2026)
- [Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective](https://mlsanity.com/paper/2607.18026/) (2026)
- [MeRoTune: RoPE-Safe Merging with a Tunable Dial](https://mlsanity.com/paper/2609.07971/) (2026)
- [SLIM: Simplex-Lattice Interpolation Merging](https://mlsanity.com/paper/2610.01037/) (2026)
- [Real-Time Joint Audio-Video Generation by Parallel Adapter Composition](https://mlsanity.com/paper/2610.10343/) (2026)

## Abstract (arXiv)

Model merging combines the parameters of multiple neural networks into a single model without additional training. As fine-tuned large language models (LLMs) proliferate, merging offers a computationally efficient alternative to ensembles and full retraining, enabling practitioners to compose specialized capabilities at minimal cost. This survey examines model merging in the LLM era through the FUSE taxonomy, organized along Foundations, Unification Strategies, Scenarios, and Ecosystem. We first establish the theoretical underpinnings of merging, including loss landscape geometry and mode connectivity, then systematically review the algorithmic space spanning weight averaging, task vector arithmetic, sparsification-enhanced methods, mixture-of-experts architectures, and evolutionary optimization. We further examine downstream applications across multi-task learning, safety alignment, domain specialization, and federated learning, and survey the supporting ecosystem of tools and evaluation benchmarks. Finally, we identify key open challenges and future directions, aiming to equip researchers and practitioners with a structured foundation for advancing model merging.