Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Unified Vision-Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we introduce the interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design a two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by reinforcement learning to promote adaptive and autonomous control. Extensive experiments demonstrate that AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, with strong transferability to various UVLMs architectures. Visual analyses further validate the effectiveness of implicit visual thoughts. These results highlight AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation. The project page is at https://sqwu.top/AD-Loop.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 72 of its 82 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
72 of 82 references.
Closest ancestors · 23
Same direction, newest first
- 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning30 Jul
- 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
- 2025Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought29 May
- 2025Thinking with Generated Images28 May
- 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO19 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT1 May
- 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
11 more
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model21 Jan
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
- 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models17 Dec
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
Other references · 49
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2023Qwen Technical Reportcited by 3,273
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
37 more
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2024OmniGen: Unified Image Generationcited by 388
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
- 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
- 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
- 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
- 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
- 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
- 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
- 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
- 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 64
- 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
- 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
- 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
- 2024Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Headscited by 44
- 20241.58-bit FLUXcited by 31
- 2025On Path to Multimodal Generalist: General-Level and General-Benchcited by 18
- 2025Co-Reinforcement Learning for Unified Multimodal Understanding and Generationcited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
- 2026Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
- 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
- 2026Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
- 2025Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning