mlsanityData to 14 Oct 2026

Images Talk Back: Guiding MMDiTs by Suppressing Image-to-Text Feedback

arXiv 2610.13941 ↗First posted 9 Oct 2026cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Multimodal diffusion transformers (MMDiTs) allow the evolving image to update internal text representations through joint attention. These representations then guide subsequent image updates, raising a question: does this feedback help correct deviations from the prompt, or preserve them? We find that image-to-text (I2T) feedback can preserve the objects and attributes already represented in the evolving image, including errors. Transplanting I2T messages can rescue failures or disrupt successes. At matched intervention strengths, suppressing the influence of I2T feedback improves generation, whereas opposite controls degrade it. These findings motivate Semantic Feedback Guidance (SFG), which suppresses the influence of I2T feedback by guiding away from an auxiliary branch that strengthens this feedback and weakens text-to-image conditioning. The in-place variant, iSFG, directly suppresses I2T feedback and strengthens text-to-image conditioning, requiring no extra model evaluations. Temporal interventions motivate running the SFG branch during the first 20% of image sampling steps, requiring about 1.2× the base NFE without classifier-free guidance (CFG). Experiments across three image-generation backbones show gains in alignment and preference-model scores. On FLUX, adding SFG to CFG raises GenEval from 0.6661 to 0.7479; on SD3.5M, iSFG adds +1.33 HPSv2.1 over CFG at zero extra NFE. The control also transfers to text-to-video and image-to-video generation, improving VBench scores. Code is available at https://github.com/yyc1215lll-creator/SFG.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 30 of its 32 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

30 of 32 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers21 Jul
  2. 2026Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion Transformers for Text-to-Image Generation6 Feb
  3. 2025Exploring Multimodal Diffusion Transformers for Enhanced Prompt-based Image Editing11 Aug
  4. 2025Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers9 Jun
  5. 2025Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis15 May
  6. 2025Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models12 Jan
  7. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  8. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan

Other references · 22

  1. 2022Classifier-Free Diffusion Guidancecited by 5,847
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  5. 2025Qwen-Image Technical Reportcited by 1,044
  6. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  7. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 299
  8. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 260
  9. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 254
  10. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 154
  11. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Modelscited by 149
  12. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 116
10 more
  1. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidancecited by 113
  2. 2024Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attentioncited by 59
  3. 2025CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Modelscited by 52
  4. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Modelscited by 36
  5. 20253DIS-FLUX: simple and efficient multi-instance generation with DiT renderingcited by 21
  6. 2026Guiding a Diffusion Model by Swapping Its Tokenscited by 3
  7. 2026DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracingcited by 3
  8. 2026Diagnosing and Correcting Concept Omission in Multimodal Diffusion Transformerscited by 2
  9. 2026GuidedBridge: Training-freely Improving Bridge Models with Prior Guidancecited by 2
  10. 2026Vision-Language Binding in In-Context Image Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers
  2. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  3. 2023Semantic Guidance Tuning for Text-To-Image Diffusion Models
  4. 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting
  5. 2025Rare Text Semantics Were Always There in Your Diffusion Transformer
  6. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
  7. 2026Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation
  8. 2026AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flow