mlsanityData to 14 Oct 2026

OminiControl2: Efficient Conditioning for Diffusion Transformers

arXiv 2503.08280 ↗First posted 11 Mar 2025cs.CV cs.AIcited by 57

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Fine-grained control of text-to-image diffusion transformer models (DiT) remains a critical challenge for practical deployment. While recent advances such as OminiControl and others have enabled a controllable generation of diverse control signals, these methods face significant computational inefficiency when handling long conditional inputs. We present OminiControl2, an efficient framework that achieves efficient image-conditional image generation. OminiControl2 introduces two key innovations: (1) a dynamic compression strategy that streamlines conditional inputs by preserving only the most semantically relevant tokens during generation, and (2) a conditional feature reuse mechanism that computes condition token features only once and reuses them across denoising steps. These architectural improvements preserve the original framework's parameter efficiency and multi-modal versatility while dramatically reducing computational costs. Our experiments demonstrate that OminiControl2 reduces conditional processing overhead by over 90% compared to its predecessor, achieving an overall 5.9× speedup in multi-conditional generation scenarios. This efficiency enables the practical implementation of complex, multi-modal control for high-quality image synthesis with DiT models.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

33 of 39 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing28 Feb
  2. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generation27 Nov
  3. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback11 Apr
  4. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  5. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb

Other references · 26

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  8. 2022Token Merging: Your ViT But Fastercited by 1,021
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  10. 2022Efficiently Scaling Transformer Inferencecited by 539
  11. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  12. 2023Token Merging for Fast Stable Diffusioncited by 224
14 more
  1. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 175
  2. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
  3. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  4. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
  5. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
  6. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 133
  7. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  8. 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
  9. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformerscited by 37
  10. 2020Multi-modal Attention for Speech Emotion Recognitioncited by 29
  11. 2025RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformerscited by 18
  12. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  13. 2025UniCP: A Unified Caching and Pruning Framework for Efficient Video Generationcited by 14
  14. 2025CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformerscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 29; 2026: 28.

57 papers.

Sort:

By area:

  1. 2025OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Datacited by 32
  2. 2025FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformerscited by 18
  3. 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 18
  4. 2025TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesiscited by 13
  5. 2025OmniTry: Virtual Try-On Anything without Maskscited by 13
  6. 2025FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Modelscited by 11
  7. 2025Image Editing As Programs with Diffusion Modelscited by 10
  8. 2025IC-Custom: Diverse Image Customization via In-Context Learningcited by 10
  9. 2025LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformercited by 7
  10. 2025JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-oncited by 7
  11. 2025DeContext as Defense: Safe Image Editing in Diffusion Transformerscited by 7
  12. 2025SpotEdit: Selective Region Editing in Diffusion Transformerscited by 7
38 more
  1. 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 6
  2. 2025LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editingcited by 5
  3. 2025ConsistCompose: Unified Multimodal Layout Control for Image Compositioncited by 5
  4. 2025NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generationcited by 5
  5. 2026AVControl: Efficient Framework for Training Audio-Visual Controlscited by 5
  6. 2025Neural-Driven Image Editingcited by 4
  7. 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
  8. 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigmcited by 4
  9. 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
  10. 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
  11. 2025Generative Refocusing: Flexible Defocus Control from a Single Imagecited by 4
  12. 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
  13. 2025InfiniHuman: Infinite 3D Human Creation with Precise Controlcited by 3
  14. 2025DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detectioncited by 3
  15. 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generationcited by 3
  16. 2026Inject Where It Matters: Training-Free Spatially-Adaptive Identity Preservation for Text-to-Image Personalizationcited by 3
  17. 2025Vision Bridge Transformer at Scalecited by 2
  18. 2026RSEdit: Text-Guided Image Editing for Remote Sensingcited by 2
  19. 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformercited by 2
  20. 2026Exploring the Performance Frontier of Compact Unified Image Generation Modelscited by 2
  21. 2025A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Spacecited by 1
  22. 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Modelscited by 1
  23. 2025ContextDrag: Precise Drag-Based Image Editing via Context-Preserving Token Injection and Position-Aligned Attentioncited by 1
  24. 2026FlowFixer: Towards Detail-Preserving Subject-Driven Generationcited by 1
  25. 2026Unsafe2Safe: Controllable Image Anonymization for Downstream Utilitycited by 1
  26. 2026TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolutioncited by 1
  27. 2026Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulationcited by 1
  28. 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generationcited by 0
  29. 2025A Practical Investigation of Spatially-Controlled Image Generation with Transformerscited by 0
  30. 2026Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scopingcited by 0
  31. 2026PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-Oncited by 0
  32. 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvascited by 0
  33. 2026Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generationcited by 0
  34. 2026Adaptive Subspace Projection for Generative Personalizationcited by 0
  35. 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editingcited by 0
  36. 2026Thinking in Boxes: 3D Editing in Real Images Made Easycited by 0
  37. 2026Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfercited by 0
  38. 2026Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creationcited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025NanoControl: A Lightweight Framework for Precise and Efficient Control in Diffusion Transformer
  2. 2024ECNet: Effective Controllable Text-to-Image Diffusion Models
  3. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer
  4. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  5. 2024DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation
  6. 2024A Simple Approach to Unifying Diffusion-based Conditional Generation
  7. 2024CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation
  8. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation