From "What" to "How": Constrained Reasoning for Autoregressive Image Generation
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Autoregressive image generation has seen recent improvements with the introduction of chain-of-thought and reinforcement learning. However, current methods merely specify "What" details to depict by rewriting the input prompt, yet fundamentally fail to reason about "How" to structure the overall image. This inherent limitation gives rise to persistent issues, such as spatial ambiguity directly causing unrealistic object overlaps. To bridge this gap, we propose CoR-Painter, a novel framework that pioneers a "How-to-What" paradigm by introducing Constrained Reasoning to guide the autoregressive generation. Specifically, it first deduces "How to draw" by deriving a set of visual constraints from the input prompt, which explicitly govern spatial relationships, key attributes, and compositional rules. These constraints steer the subsequent generation of a detailed description "What to draw", providing a structurally sound and coherent basis for accurate visual synthesis. Additionally, we introduce a Dual-Objective GRPO strategy that specifically optimizes the textual constrained reasoning and visual projection processes to ensure the coherence and quality of the entire generation pipeline. Extensive experiments on T2I-CompBench, GenEval, and WISE demonstrate that our method achieves state-of-the-art performance, with significant improvements in spatial metrics (e.g., +5.41% on T2I-CompBench).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 40 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
34 of 40 references.
Closest ancestors · 4
Same direction, newest first
- 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning22 May
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT1 May
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step23 Jan
Other references · 30
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2024Qwen2 Technical Reportcited by 2,120
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
18 more
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
- 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 155
- 2021Simple multi-dataset detectioncited by 127
- 2025GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generationcited by 57
- 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 57
- 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 28
- 2024Scalable Autoregressive Image Generation with Mambacited by 26
- 2025Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Modelscited by 18
- 2025Co-Reinforcement Learning for Unified Multimodal Understanding and Generationcited by 15
- 2025FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RLcited by 13
- 2025Improving Autoregressive Image Generation through Coarse-to-Fine Token Predictioncited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Improving Chain-of-Thought Efficiency for Autoregressive Image Generation
- 2025ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL
- 2025Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation
- 2025Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling
- 2026Large Language Models are Universal Reasoners for Visual Generation
- 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation
- 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Again