mlsanityData to 14 Oct 2026

X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model

arXiv 2312.02238 ↗First posted 4 Dec 2023cs.CV cs.AI cs.MMcited by 19

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We introduce X-Adapter, a universal upgrader to enable the pretrained plug-and-play modules (e.g., ControlNet, LoRA) to work directly with the upgraded text-to-image diffusion model (e.g., SDXL) without further retraining. We achieve this goal by training an additional network to control the frozen upgraded model with the new text-image data pairs. In detail, X-Adapter keeps a frozen copy of the old model to preserve the connectors of different plugins. Additionally, X-Adapter adds trainable mapping layers that bridge the decoders from models of different versions for feature remapping. The remapped features will be used as guidance for the upgraded model. To enhance the guidance ability of X-Adapter, we employ a null-text training strategy for the upgraded model. After training, we also introduce a two-stage denoising strategy to align the initial latents of X-Adapter and the upgraded model. Thanks to our strategies, X-Adapter demonstrates universal compatibility with various plugins and also enables plugins of different versions to work together, thereby expanding the functionalities of diffusion community. To verify the effectiveness of the proposed method, we conduct extensive experiments and the results show that X-Adapter may facilitate wider application in the upgraded foundational diffusion model.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 27 of its 37 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

27 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  2. 2023Inserting Anybody in Diffusion Models via Celeb Basis1 Jun
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
7 more
  1. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  2. 2021Variational Diffusion Modelscited by 1,153
  3. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  4. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  5. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  6. 2023Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigmcited by 5
  7. 2023TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adaptercited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 9; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  2. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 66
  3. 2024ControlVAR: Exploring Controllable Visual Autoregressive Modelingcited by 58
  4. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 45
  5. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  6. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  7. 2024CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generationcited by 21
  8. 2025SCALAR: Scale-wise Controllable Visual Autoregressive Learningcited by 18
  9. 2024MUSCLE: A Model Update Strategy for Compatible LLM Evolutioncited by 15
  10. 2025LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptationcited by 11
  11. 2025TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Visioncited by 9
  12. 2025Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Modelscited by 3
7 more
  1. 2026Exploring Data-Free LoRA Transferability for Video Diffusion Modelscited by 3
  2. 2025Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Abilitycited by 2
  3. 2025Glance: Accelerating Diffusion Models with 1 Samplecited by 2
  4. 2026Enhancing Hands in 3D Whole-Body Pose Estimation with Conditional Hands Modulatorcited by 2
  5. 2025Heeding the Inner Voice: Aligning ControlNet Training via Intermediate Features Feedbackcited by 1
  6. 2024No Re-Train, More Gain: Upgrading Backbones with Diffusion model for Pixel-Wise and Weakly-Supervised Few-Shot Segmentationcited by 0
  7. 2026DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Restoration Adapter for Real-World Image Restoration
  2. 2022Back to the Source: Diffusion-Driven Test-Time Adaptation
  3. 2024Stylus: Automatic Adapter Selection for Diffusion Models
  4. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models
  5. 2025UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models
  6. 2025BIR-Adapter: A parameter-efficient diffusion adapter for blind image restoration
  7. 2024Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising
  8. 2025Att-Adapter: A Robust and Precise Domain-Specific Multi-Attributes T2I Diffusion Adapter via Conditional Variational Autoencoder