# CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging

arXiv: https://arxiv.org/abs/2505.06977 · HTML: https://mlsanity.com/paper/2505.06977/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 64 references that are in the map.
- First posted on arXiv 11 May 2025.

## Builds on (most cited first, 30 of 49)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [TextCaps: a Dataset for Image Captioning with Reading Comprehension](https://mlsanity.com/paper/2003.12462/) (2020)
- [Dataless Knowledge Fusion by Merging Weights of Language Models](https://mlsanity.com/paper/2212.09849/) (2022)
- [AdaMerging: Adaptive Model Merging for Multi-Task Learning](https://mlsanity.com/paper/2310.02575/) (2023)
- [Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models](https://mlsanity.com/paper/2305.12827/) (2023)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://mlsanity.com/paper/2010.06808/) (2020)
- [Mixture of LoRA Experts](https://mlsanity.com/paper/2404.13628/) (2024)
- [Composing Parameter-Efficient Modules with Arithmetic Operations](https://mlsanity.com/paper/2306.14870/) (2023)

## Built on by (most cited first, 23 of 23)

- [MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent](https://mlsanity.com/paper/2511.18810/) (2025)
- [Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration](https://mlsanity.com/paper/2505.23859/) (2025)
- [RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior](https://mlsanity.com/paper/2508.03140/) (2025)
- [When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Merging](https://mlsanity.com/paper/2602.05536/) (2026)
- [Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging](https://mlsanity.com/paper/2512.01461/) (2025)
- [StatsMerging: Statistics-Guided Model Merging via Task-Specific Teacher Distillation](https://mlsanity.com/paper/2506.04567/) (2025)
- [Learning Not To Interfere: Adapting Experts For Model Merging](https://mlsanity.com/paper/2603.13467/) (2026)
- [Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem](https://mlsanity.com/paper/2604.14808/) (2026)
- [Dynamic Model Merging Made Slim](https://mlsanity.com/paper/2605.18904/) (2026)
- [Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing](https://mlsanity.com/paper/2606.16501/) (2026)
- [Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation](https://mlsanity.com/paper/2606.19164/) (2026)
- [Exploring Information Seeking Agent Consolidation](https://mlsanity.com/paper/2602.00585/) (2026)
- [LARV: Data-Free Layer-wise Adaptive Rescaling Veneer for Model Merging](https://mlsanity.com/paper/2602.09413/) (2026)
- [Domain-Adaptive Model Merging Across Disconnected Modes](https://mlsanity.com/paper/2603.05957/) (2026)
- [Model Merging via Data-Free Covariance Estimation](https://mlsanity.com/paper/2604.01329/) (2026)
- [MAny: Merge Anything for Multimodal Continual Instruction Tuning](https://mlsanity.com/paper/2604.14016/) (2026)
- [Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?](https://mlsanity.com/paper/2605.23913/) (2026)
- [Saliency-Aware Model Merging](https://mlsanity.com/paper/2606.00511/) (2026)
- [On the Vulnerability of Parameter-Level Defenses to Model Merging](https://mlsanity.com/paper/2606.30360/) (2026)
- [Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shifts](https://mlsanity.com/paper/2607.00666/) (2026)
- [Signature-Guided Capacity Occupancy for Dense Expert Merging](https://mlsanity.com/paper/2608.09201/) (2026)
- [Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface](https://mlsanity.com/paper/2609.33125/) (2026)
- [USA: Update-aware SAM for Cross-domain On-Policy Disitllation of Language Agents](https://mlsanity.com/paper/2609.34225/) (2026)

## Abstract (arXiv)

Multi-task model merging offers a promising paradigm for integrating multiple expert models into a unified model without additional training. Existing state-of-the-art techniques, such as Task Arithmetic and its variants, merge models by accumulating task vectors -- the parameter differences between pretrained and finetuned models. However, task vector accumulation is often hindered by knowledge conflicts, leading to performance degradation. To address this challenge, we propose Conflict-Aware Task Merging (CAT Merging), a novel training-free framework that selectively trims conflict-prone components from the task vectors. CAT Merging introduces several parameter-specific strategies, including projection for linear weights and masking for scaling and shifting parameters in normalization layers. Extensive experiments on vision, language, and vision-language tasks demonstrate that CAT Merging effectively suppresses knowledge conflicts, achieving average accuracy improvements of up to 2.5% (ViT-B/32) and 2.0% (ViT-L/14) over state-of-the-art methods.