# Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging

arXiv: https://arxiv.org/abs/2408.12105 · HTML: https://mlsanity.com/paper/2408.12105/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2024.

## Built on by (most cited first, 14 of 14)

- [PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model](https://mlsanity.com/paper/2505.06274/) (2025)
- [Label-Free Cross-Task LoRA Merging with Null-Space Compression](https://mlsanity.com/paper/2603.26317/) (2026)
- [COPA: Comparing the incomparable in multi-objective model evaluation](https://mlsanity.com/paper/2503.14321/) (2025)
- [UniARM: Towards a Unified Autoregressive Reward Model for Multi-Objective Test-Time Alignment](https://mlsanity.com/paper/2602.09538/) (2026)
- [VeFA: Vector-Based Feature Space Adaptation for Robust Model Fine-Tuning](https://mlsanity.com/paper/2510.19155/) (2025)
- [Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy](https://mlsanity.com/paper/2603.26299/) (2026)
- [StatsMerging: Statistics-Guided Model Merging via Task-Specific Teacher Distillation](https://mlsanity.com/paper/2506.04567/) (2025)
- [ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models](https://mlsanity.com/paper/2512.06515/) (2025)
- [Certifying Model Upgrades with Slice-Wise Non-Regression and Incumbent Fallback](https://mlsanity.com/paper/2609.13714/) (2026)
- [Interpretability by Design for Efficient Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2506.04022/) (2025)
- [SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Front](https://mlsanity.com/paper/2605.20619/) (2026)
- [Multi-Objective Exploration and Preference Optimization via Mutual Information](https://mlsanity.com/paper/2607.01392/) (2026)
- [More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility](https://mlsanity.com/paper/2607.27255/) (2026)
- [The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization](https://mlsanity.com/paper/2609.39055/) (2026)

## Abstract (arXiv)

Model merging, which combines multiple models into a single model, has gained popularity in recent years. By efficiently integrating the capabilities of various models, this significantly reduces the parameter count and memory usage. However, current methods can only produce one single merged model. This necessitates a performance trade-off due to conflicts among the various models, and the resultant one-size-fits-all model may not align with the preferences of different users who may prioritize certain models over others. To address this issue, we propose preference-aware model merging, and formulate this as a multi-objective optimization problem in which the performance of the merged model on each base model's task is treated as an objective. In a single merging process, the proposed parameter-efficient structure generates a Pareto set of merged models, with each representing a Pareto-optimal solution for a preference. Users can then select merged models tailored to their preferences from this learned Pareto set. Experimental results demonstrate that the proposed Pareto Merging produces diverse trade-off models and achieves higher test accuracy compared to state-of-the-art merging baselines.