GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
Fine-tuning-based adaptation is widely used to customize diffusion-based image generation, leading to large collections of community-created adapters that capture diverse subjects and styles. Adapters derived from the same base model can be merged with weights, enabling the synthesis of new visual results within a vast and continuous design space. To explore this space, current workflows rely on manual slider-based tuning, an approach that scales poorly and makes weight selection difficult, even when the candidate set is limited to 20-30 adapters. We propose GimmBO to support interactive exploration of adapter merging for image generation through Preferential Bayesian Optimization (PBO). Motivated by observations from real-world usage, including sparsity and constrained weight ranges, we introduce a two-stage BO backend that improves sampling efficiency and convergence in high-dimensional spaces. We evaluate our approach with simulated users and a user study, demonstrating improved convergence, high success rates, and consistent gains over BO and line-search baselines, and further show the flexibility of the framework through several extensions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 45 of its 78 references that are in the map. First posted on arXiv 26 Jan 2026.
Builds on
45 of 78 references.
References · 45
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
- 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
- 2021Merging Models with Fisher-Weighted Averagingcited by 621
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
- 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
33 more
- 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
- 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
- 2024InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generationcited by 170
- 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 141
- 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
- 2024Measuring Style Similarity in Diffusion Modelscited by 109
- 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
- 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
- 2017The reparameterization trick for acquisition functionscited by 63
- 2023PromptMagician: Interactive Prompt Engineering for Text-to-Image Creationcited by 58
- 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
- 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
- 2023StyleAdapter: A Unified Stylized Image Generation Modelcited by 39
- 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
- 2021TREGO: a Trust-Region Framework for Efficient Global Optimizationcited by 29
- 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
- 2020Sequential Gallery for Interactive Visual Design Optimizationcited by 25
- 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
- 2024Stylus: Automatic Adapter Selection for Diffusion Modelscited by 18
- 2015Projection onto the capped simplexcited by 15
- 2020High Dimensional Bayesian Optimization Assisted by Principal Component Analysiscited by 14
- 2022GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Informationcited by 13
- 2024Subject-driven Text-to-Image Generation via Preference-based Reinforcement Learningcited by 12
- 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformerscited by 12
- 2024PrompTHis: Visualizing the Process and Influence of Prompt Editing during Text-to-Image Creationcited by 11
- 2025PromptMap: An Alternative Interaction Style for AI-Based Image Generationcited by 8
- 2024Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Modelscited by 7
- 2024A LoRA is Worth a Thousand Picturescited by 7
- 2025FontCraft: Multimodal Font Design Using Interactive Bayesian Optimizationcited by 7
- 2020Generative Melody Composition with Human-in-the-Loop Bayesian Optimizationcited by 6
- 2021AdaptiFont: Increasing Individuals' Reading Speed with a Generative Font Model and Bayesian Optimizationcited by 5
- 2025Cooperative Design Optimization through Natural Language Interactioncited by 4
- 2025LoRAverse: A Submodular Framework to Retrieve Diverse Adapters for Diffusion Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Personalized Image Generation via Human-in-the-loop Bayesian Optimization
- 2025RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
- 2022Towards Practical Plug-and-Play Diffusion Models
- 2023Conditional Image Generation with Pretrained Generative Model
- 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
- 2022Elucidating the Design Space of Diffusion-Based Generative Models
- 2022Accelerating Score-based Generative Models for High-Resolution Image Synthesis
- 2024Alfie: Democratising RGBA Image Generation With No