mlsanityData to 14 Oct 2026

Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models

arXiv 2406.03683 ↗First posted 6 Jun 2024cs.LG stat.MLcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We propose a Bayesian framework for fine-tuning large diffusion models with a novel network structure called Bayesian Power Steering (BPS). We clarify the meaning behind adaptation from a textitlarge probability space to a textitsmall probability space and explore the task of fine-tuning pre-trained models using learnable modules from a Bayesian perspective. BPS extracts task-specific knowledge from a pre-trained model's learned prior distribution. It efficiently leverages large diffusion models, differentially intervening different hidden features with a head-heavy and foot-light configuration. Experiments highlight the superiority of BPS over contemporary methods across a range of tasks even with limited amount of data. Notably, BPS attains an FID score of 10.49 under the sketch condition on the COCO17 dataset.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 49 of its 59 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

49 of 59 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
28 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  5. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  7. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  8. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  9. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  10. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  11. 2015Holistically-Nested Edge Detectioncited by 592
  12. 2023Zero-shot Image-to-Image Translationcited by 505
  13. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  14. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  15. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  16. 2022Brain Imaging Generation with Latent Diffusion Modelscited by 268
  17. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
  18. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  19. 2021Pixel Difference Networks for Efficient Edge Detectioncited by 168
  20. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 130
  21. 2023Differentially Private Diffusion Models Generate Useful Synthetic Imagescited by 86
  22. 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generationcited by 65
  23. 2023Harnessing large-language models to generate private synthetic textcited by 65
  24. 2023DP-LDMs: Differentially Private Latent Diffusion Modelscited by 29
  25. 2022Pre-trained Perceptual Features Improve Differentially Private Image Generationcited by 28
  26. 2023Augmenting medical image classifiers with synthetic data from latent diffusion modelscited by 22
  27. 2023Differentially Private Image Classification by Learning Priors from Random Processescited by 19
  28. 2023Conditional Stochastic Interpolation for Generative Learningcited by 9

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Boosting Statistic Learning with Synthetic Data from Pretrained Large Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Models
  2. 2026Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning
  3. 2026MMD Guidance: Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidance
  4. 2023A Closer Look at Parameter-Efficient Tuning in Diffusion Models
  5. 2025FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning
  6. 2025Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model
  7. 2024Diffusion Tuning: Transferring Diffusion Models via Chain of Forgetting
  8. 2024FINE: Factorizing Knowledge for Initialization of Variable-sized Diffusion Models