OminiControl2: Efficient Conditioning for Diffusion Transformers
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Fine-grained control of text-to-image diffusion transformer models (DiT) remains a critical challenge for practical deployment. While recent advances such as OminiControl and others have enabled a controllable generation of diverse control signals, these methods face significant computational inefficiency when handling long conditional inputs. We present OminiControl2, an efficient framework that achieves efficient image-conditional image generation. OminiControl2 introduces two key innovations: (1) a dynamic compression strategy that streamlines conditional inputs by preserving only the most semantically relevant tokens during generation, and (2) a conditional feature reuse mechanism that computes condition token features only once and reuses them across denoising steps. These architectural improvements preserve the original framework's parameter efficiency and multi-modal versatility while dramatically reducing computational costs. Our experiments demonstrate that OminiControl2 reduces conditional processing overhead by over 90% compared to its predecessor, achieving an overall 5.9× speedup in multi-conditional generation scenarios. This efficiency enables the practical implementation of complex, multi-modal control for high-quality image synthesis with DiT models.
Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 11 Mar 2025.
Builds on
33 of 39 references.
Closest ancestors · 7
Same direction, newest first
- 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing28 Feb
- 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generation27 Nov
- 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback11 Apr
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
- 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
Other references · 26
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2022Token Merging: Your ViT But Fastercited by 1,021
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2022Efficiently Scaling Transformer Inferencecited by 539
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2023Token Merging for Fast Stable Diffusioncited by 224
14 more
- 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 175
- 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
- 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
- 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 133
- 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
- 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
- 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformerscited by 37
- 2020Multi-modal Attention for Speech Emotion Recognitioncited by 29
- 2025RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformerscited by 18
- 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
- 2025UniCP: A Unified Caching and Pruning Framework for Efficient Video Generationcited by 14
- 2025CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformerscited by 11
Built on by
Citing papers per year: 2025: 29; 2026: 28.
57 papers.
Sort:
By area:
- 2025OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Datacited by 32
- 2025FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformerscited by 18
- 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 18
- 2025TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesiscited by 13
- 2025OmniTry: Virtual Try-On Anything without Maskscited by 13
- 2025FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Modelscited by 11
- 2025Image Editing As Programs with Diffusion Modelscited by 10
- 2025IC-Custom: Diverse Image Customization via In-Context Learningcited by 10
- 2025LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformercited by 7
- 2025JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-oncited by 7
- 2025DeContext as Defense: Safe Image Editing in Diffusion Transformerscited by 7
- 2025SpotEdit: Selective Region Editing in Diffusion Transformerscited by 7
38 more
- 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 6
- 2025LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editingcited by 5
- 2025ConsistCompose: Unified Multimodal Layout Control for Image Compositioncited by 5
- 2025NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generationcited by 5
- 2026AVControl: Efficient Framework for Training Audio-Visual Controlscited by 5
- 2025Neural-Driven Image Editingcited by 4
- 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
- 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigmcited by 4
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
- 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
- 2025Generative Refocusing: Flexible Defocus Control from a Single Imagecited by 4
- 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
- 2025InfiniHuman: Infinite 3D Human Creation with Precise Controlcited by 3
- 2025DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detectioncited by 3
- 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generationcited by 3
- 2026Inject Where It Matters: Training-Free Spatially-Adaptive Identity Preservation for Text-to-Image Personalizationcited by 3
- 2025Vision Bridge Transformer at Scalecited by 2
- 2026RSEdit: Text-Guided Image Editing for Remote Sensingcited by 2
- 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformercited by 2
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Modelscited by 2
- 2025A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Spacecited by 1
- 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Modelscited by 1
- 2025ContextDrag: Precise Drag-Based Image Editing via Context-Preserving Token Injection and Position-Aligned Attentioncited by 1
- 2026FlowFixer: Towards Detail-Preserving Subject-Driven Generationcited by 1
- 2026Unsafe2Safe: Controllable Image Anonymization for Downstream Utilitycited by 1
- 2026TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolutioncited by 1
- 2026Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulationcited by 1
- 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generationcited by 0
- 2025A Practical Investigation of Spatially-Controlled Image Generation with Transformerscited by 0
- 2026Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scopingcited by 0
- 2026PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-Oncited by 0
- 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvascited by 0
- 2026Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generationcited by 0
- 2026Adaptive Subspace Projection for Generative Personalizationcited by 0
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editingcited by 0
- 2026Thinking in Boxes: 3D Editing in Real Images Made Easycited by 0
- 2026Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfercited by 0
- 2026Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creationcited by 0
50 most cited of 57 All 57 (JSON)
- 2026RefAdapt-DiT: Adaptive Joint Attention for Reference-Conditioned Diffusion Transformers26 Sep 2026
- 2026OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction24 Sep 2026
- 2026GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioning15 Sep 2026
- 2026Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching14 Sep 2026
- 2026UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation11 Sep 2026
- 2026ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features4 Sep 2026
- 2026Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation21 Aug 2026
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Models20 Aug 2026
- 2026Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation19 Aug 2026
- 2026Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation7 Jul 2026
- 2026Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer30 Jun 2026
- 2026Thinking in Boxes: 3D Editing in Real Images Made Easy18 Jun 2026
38 more
- 2026TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolution15 Jun 2026
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing17 May 2026
- 2026Adaptive Subspace Projection for Generative Personalization8 May 2026
- 2026Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation1 Apr 2026
- 2026Unsafe2Safe: Controllable Image Anonymization for Downstream Utility30 Mar 2026
- 2026AVControl: Efficient Framework for Training Audio-Visual Controls25 Mar 2026
- 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer16 Mar 2026
- 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvas16 Mar 2026
- 2026RSEdit: Text-Guided Image Editing for Remote Sensing14 Mar 2026
- 2026PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On12 Mar 2026
- 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation6 Mar 2026
- 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation26 Feb 2026
- 2026FlowFixer: Towards Detail-Preserving Subject-Driven Generation24 Feb 2026
- 2026Inject Where It Matters: Training-Free Spatially-Adaptive Identity Preservation for Text-to-Image Personalization15 Feb 2026
- 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generation7 Feb 2026
- 2026Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping6 Feb 2026
- 2025SpotEdit: Selective Region Editing in Diffusion Transformers26 Dec 2025
- 2025Generative Refocusing: Flexible Defocus Control from a Single Image18 Dec 2025
- 2025DeContext as Defense: Safe Image Editing in Diffusion Transformers18 Dec 2025
- 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learning17 Dec 2025
- 2025ContextDrag: Precise Drag-Based Image Editing via Context-Preserving Token Injection and Position-Aligned Attention9 Dec 2025
- 2025NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generation4 Dec 2025
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues2 Dec 2025
- 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Models1 Dec 2025
- 2025Vision Bridge Transformer at Scale28 Nov 2025
- 2025DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detection24 Nov 2025
- 2025ConsistCompose: Unified Multimodal Layout Control for Image Composition23 Nov 2025
- 2025A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space13 Nov 2025
- 2025InfiniHuman: Infinite 3D Human Creation with Precise Control13 Oct 2025
- 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generation13 Oct 2025
- 2025LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editing30 Sep 2025
- 2025FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models28 Aug 2025
- 2025JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-on25 Aug 2025
- 2025OmniTry: Virtual Try-On Anything without Masks19 Aug 2025
- 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigm5 Aug 2025
- 2025LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformer1 Aug 2025
- 2025A Practical Investigation of Spatially-Controlled Image Generation with Transformers21 Jul 2025
- 2025PositionIC: Unified Position and Identity Consistency for Image Customization18 Jul 2025
50 newest of 57 All 57 (JSON)
- 2025OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Datacited by 32
- 2025FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformerscited by 18
- 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 18
- 2025FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Modelscited by 11
- 2025Image Editing As Programs with Diffusion Modelscited by 10
- 2025LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformercited by 7
- 2025JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-oncited by 7
- 2025DeContext as Defense: Safe Image Editing in Diffusion Transformerscited by 7
- 2025SpotEdit: Selective Region Editing in Diffusion Transformerscited by 7
- 2025LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editingcited by 5
- 2025Neural-Driven Image Editingcited by 4
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
12 most cited of 23 All 23 (JSON)
- 2026RefAdapt-DiT: Adaptive Joint Attention for Reference-Conditioned Diffusion Transformers26 Sep 2026
- 2026GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioning15 Sep 2026
- 2026Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching14 Sep 2026
- 2026ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features4 Sep 2026
- 2026Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation21 Aug 2026
- 2026Mise-en-Sc`ene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation19 Aug 2026
- 2026TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolution15 Jun 2026
- 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer16 Mar 2026
- 2026PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On12 Mar 2026
- 2025SpotEdit: Selective Region Editing in Diffusion Transformers26 Dec 2025
- 2025DeContext as Defense: Safe Image Editing in Diffusion Transformers18 Dec 2025
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues2 Dec 2025
12 newest of 23 All 23 (JSON)
- 2025TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesiscited by 13
- 2025OmniTry: Virtual Try-On Anything without Maskscited by 13
- 2025IC-Custom: Diverse Image Customization via In-Context Learningcited by 10
- 2026InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generationcited by 6
- 2025ConsistCompose: Unified Multimodal Layout Control for Image Compositioncited by 5
- 2025NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generationcited by 5
- 2026AVControl: Efficient Framework for Training Audio-Visual Controlscited by 5
- 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
- 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigmcited by 4
- 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
- 2025Generative Refocusing: Flexible Defocus Control from a Single Imagecited by 4
- 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
12 most cited of 33 All 33 (JSON)
- 2026OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction24 Sep 2026
- 2026UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation11 Sep 2026
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Models20 Aug 2026
- 2026Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer30 Jun 2026
- 2026Thinking in Boxes: 3D Editing in Real Images Made Easy18 Jun 2026
- 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing17 May 2026
- 2026Adaptive Subspace Projection for Generative Personalization8 May 2026
- 2026Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation1 Apr 2026
- 2026Unsafe2Safe: Controllable Image Anonymization for Downstream Utility30 Mar 2026
- 2026AVControl: Efficient Framework for Training Audio-Visual Controls25 Mar 2026
- 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvas16 Mar 2026
- 2026RSEdit: Text-Guided Image Editing for Remote Sensing14 Mar 2026
12 newest of 33 All 33 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025NanoControl: A Lightweight Framework for Precise and Efficient Control in Diffusion Transformer
- 2024ECNet: Effective Controllable Text-to-Image Diffusion Models
- 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer
- 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
- 2024DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation
- 2024A Simple Approach to Unifying Diffusion-based Conditional Generation
- 2024CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation
- 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation