# I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models

arXiv: https://arxiv.org/abs/2502.10458 · HTML: https://mlsanity.com/paper/2502.10458/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)
- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [NExT-GPT: Any-to-Any Multimodal LLM](https://mlsanity.com/paper/2309.05519/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers](https://mlsanity.com/paper/2410.10629/) (2024)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)
- [Emu: Generative Pretraining in Multimodality](https://mlsanity.com/paper/2307.05222/) (2023)
- [DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation](https://mlsanity.com/paper/2406.16855/) (2024)
- [LMFusion: Adapting Pretrained Language Models for Multimodal Generation](https://mlsanity.com/paper/2412.15188/) (2024)
- [Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models](https://mlsanity.com/paper/2409.10695/) (2024)
- [Kosmos-G: Generating Images in Context with Multimodal Large Language Models](https://mlsanity.com/paper/2310.02992/) (2023)
- [X-VILA: Cross-Modality Alignment for Large Language Model](https://mlsanity.com/paper/2405.19335/) (2024)
- [Visual Lexicon: Rich Image Features in Language Space](https://mlsanity.com/paper/2412.06774/) (2024)
- [MUMU: Bootstrapping Multimodal Image Generation from Text-to-Image Data](https://mlsanity.com/paper/2406.18790/) (2024)

## Built on by (most cited first, 12 of 12)

- [ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation](https://mlsanity.com/paper/2509.18092/) (2025)
- [MentisOculi: Revealing the Limits of Reasoning with Mental Imagery](https://mlsanity.com/paper/2602.02465/) (2026)
- [HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization](https://mlsanity.com/paper/2604.20328/) (2026)
- [Unified Thinker: A General Reasoning Modular Core for Image Generation](https://mlsanity.com/paper/2601.03127/) (2026)
- [CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvas](https://mlsanity.com/paper/2510.20820/) (2025)
- [GeoLoom: High-quality Geometric Diagram Generation from Textual Input](https://mlsanity.com/paper/2512.08180/) (2025)
- [UniCUE: Unified Recognition and Generation Framework for Chinese Cued Speech Video-to-Speech Generation](https://mlsanity.com/paper/2506.04134/) (2025)
- [Fake it till You Make it: Reward Modeling as Discriminative Prediction](https://mlsanity.com/paper/2506.13846/) (2025)
- [Semantic-level Backdoor Attack against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2602.04898/) (2026)
- [GAP3D: Generative Alignment of VLM Latents to Patch-Level Embeddings for 3D Generation](https://mlsanity.com/paper/2605.28995/) (2026)
- [Evaluating Reasoning Fidelity in Visual Text Generation](https://mlsanity.com/paper/2606.04479/) (2026)
- [Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization](https://mlsanity.com/paper/2606.10902/) (2026)

## Abstract (arXiv)

This paper presents ThinkDiff, a novel alignment paradigm that empowers text-to-image diffusion models with multimodal in-context understanding and reasoning capabilities by integrating the strengths of vision-language models (VLMs). Existing multimodal diffusion finetuning methods largely focus on pixel-level reconstruction rather than in-context reasoning, and are constrained by the complexity and limited availability of reasoning-based datasets. ThinkDiff addresses these challenges by leveraging vision-language training as a proxy task, aligning VLMs with the decoder of an encoder-decoder large language model (LLM) instead of a diffusion decoder. This proxy task builds on the observation that the LLM decoder shares the same input feature space with diffusion decoders that use the corresponding LLM encoder for prompt embedding. As a result, aligning VLMs with diffusion decoders can be simplified through alignment with the LLM decoder. Without complex training and datasets, ThinkDiff effectively unleashes understanding, reasoning, and composing capabilities in diffusion models. Experiments demonstrate that ThinkDiff significantly improves accuracy from 19.2% to 46.3% on the challenging CoBSAT benchmark for multimodal in-context reasoning generation, with only 5 hours of training on 4 A100 GPUs. Additionally, ThinkDiff demonstrates exceptional performance in composing multiple images and texts into logically coherent images. Project page: https://mizhenxing.github.io/ThinkDiff.