# CALM: Consensus-Aware Localized Merging for Multi-Task Learning

arXiv: https://arxiv.org/abs/2506.13406 · HTML: https://mlsanity.com/paper/2506.13406/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2025.

## Built on by (most cited first, 8 of 8)

- [MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent](https://mlsanity.com/paper/2511.18810/) (2025)
- [Model Merging Scaling Laws in Large Language Models](https://mlsanity.com/paper/2509.24244/) (2025)
- [Model Merging in the Essential Subspace](https://mlsanity.com/paper/2602.20208/) (2026)
- [Discovering Physical Directions in Weight Space: Composing Neural PDE Experts](https://mlsanity.com/paper/2605.14546/) (2026)
- [Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging](https://mlsanity.com/paper/2512.01461/) (2025)
- [Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy](https://mlsanity.com/paper/2603.26299/) (2026)
- [DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging](https://mlsanity.com/paper/2508.01148/) (2025)
- [Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation](https://mlsanity.com/paper/2606.19164/) (2026)

## Abstract (arXiv)

Model merging aims to integrate the strengths of multiple fine-tuned models into a unified model while preserving task-specific capabilities. Existing methods, represented by task arithmetic, are typically classified into global- and local-aware methods. However, global-aware methods inevitably cause parameter interference, while local-aware methods struggle to maintain the effectiveness of task-specific details in the merged model. To address these limitations, we propose a Consensus-Aware Localized Merging (CALM) method which incorporates localized information aligned with global task consensus, ensuring its effectiveness post-merging. CALM consists of three key components: (1) class-balanced entropy minimization sampling, providing a more flexible and reliable way to leverage unsupervised data; (2) an efficient-aware framework, selecting a small set of tasks for sequential merging with high scalability; (3) a consensus-aware mask optimization, aligning localized binary masks with global task consensus and merging them conflict-free. Experiments demonstrate the superiority and robustness of our CALM, significantly outperforming existing methods and achieving performance close to traditional MTL.