Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
Not on the map. Why?
Abstract (arXiv)
Pareto fronts are useful to find good task-mixing strategies for multitask finetuning, but they are also costly to compute. To reduce costs, recent works have used existing model merging methods to help train cheap surrogate models to estimate the Pareto fronts. However, no work has yet considered designing new model-merging methods to directly, and provably, improve the quality of Pareto fronts. Here, we fill this gap by proposing a new Bayesian approach called Variational Model Merging. In this approach, existing model-merging methods are obtained as special cases of "posterior-merging" when Gaussian posteriors are used and new model-merging strategies can be derived by using non-Gaussian posteriors. Our main theoretical result is to show that more flexible posteriors necessarily yield better estimates of Pareto fronts. For instance, a Pareto front estimate obtained by merging full-Gaussian posteriors is expected to be better than that obtained by using isotropic Gaussian posteriors. We validate the theory through extensive empirical results on vision and language transformers where better Gaussian families consistently yields better or comparable Pareto fronts. Our work is a rare instance where Bayesian ideas are used to improve Pareto analysis.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 11 Dec 2024.
Builds on
44 of 71 references.
References · 44
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
32 more
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
- 2021Merging Models with Fisher-Weighted Averagingcited by 613
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
- 2023Data Selection for Language Models via Importance Resamplingcited by 322
- 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 248
- 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 218
- 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
- 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 178
- 2013A General Framework for Updating Belief Distributionscited by 172
- 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
- 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
- 2021The Bayesian Learning Rulecited by 95
- 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
- 2024Variational Learning is Effective for Large Deep Networkscited by 62
- 2022ColD Fusion: Collaborative Descent for Distributed Multitask Finetuningcited by 52
- 2019Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximationscited by 51
- 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
- 2023Self-Influence Guided Data Reweighting for Language Model Pre-trainingcited by 37
- 2020Multi-Loss Weighting with Coefficient of Variationscited by 36
- 2023MFTCoder: Boosting Code LLMs with Multitask Fine-Tuningcited by 34
- 2024Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuningcited by 31
- 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
- 2024MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximationcited by 20
- 2022FORML: Learning to Reweight Data for Fairnesscited by 12
- 2024Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuningcited by 8
Built on by
Citing papers per year: 2025: 1.
1 paper.