Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Despite remarkable progress in image quality and prompt fidelity, text-to-image (T2I) diffusion models continue to exhibit persistent "hallucinations", where generated content subtly or significantly diverges from the intended prompt semantics. While often regarded as unpredictable artifacts, we argue that these failures reflect deeper, structured misalignments within the generative process. In this work, we propose a cognitively inspired perspective that reinterprets hallucinations as trajectory drift within a latent alignment space. Empirical observations reveal that generation unfolds within a multiaxial cognitive tension field, where the model must continuously negotiate competing demands across three key critical axes: semantic coherence, structural alignment, and knowledge grounding. We then formalize this three-axis space as the Hallucination Tri-Space and introduce the Alignment Risk Code (ARC): a dynamic vector representation that quantifies real-time alignment tension during generation. The magnitude of ARC captures overall misalignment, its direction identifies the dominant failure axis, and its imbalance reflects tension asymmetry. Based on this formulation, we develop the TensionModulator (TM-ARC): a lightweight controller that operates entirely in latent space. TM-ARC monitors ARC signals and applies targeted, axis-specific interventions during the sampling process. Extensive experiments on standard T2I benchmarks demonstrate that our approach significantly reduces hallucination without compromising image quality or diversity. This framework offers a unified and interpretable approach for understanding and mitigating generative failures in diffusion-based T2I systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 13 of its 15 references that are in the map. First posted on arXiv 7 Jul 2025.
Builds on
13 of 15 references.
Closest ancestors · 5
Same direction, newest first
- 2024Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection14 Dec
- 2024Repairing Catastrophic-Neglect in Text-to-Image Diffusion Models via Attention-Guided Feature Enhancement24 Jun
- 2024Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models10 Apr
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
Other references · 8
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
- 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
- 2024Addressing Image Hallucination in Text-to-Image Generation through Factual Image Retrievalcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Hallucination as an Upper Bound: A New Perspective on Text-to-Image Evaluation
- 2024Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models
- 2026ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
- 2025Generating on Generated: An Approach Towards Self-Evolving Diffusion Models
- 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats
- 2025CritiFusion: Semantic Critique and Spectral Alignment for Faithful Text-to-Image Generation
- 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
- 2025Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models