Latent Action Control for Reasoning-Guided Unified Image Generation
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models can encode visual understanding and image generation within a shared backbone, yet understanding does not automatically translate into control: models may infer objects, relations, or knowledge cues but fail to instantiate them in the generated image. We propose Latent Action Control (LAC), which makes reasoning actionable by representing it as hidden continuous actions inside a unified generator. Given a prompt, LAC rolls out a role-structured latent trajectory for planning, internal visual drafting, diagnosis, and refinement, and injects these actions into the hidden stream that conditions flow-based generation, without producing reasoning tokens or intermediate images. Since such action trajectories are unobserved, LAC learns them through prior-guided variational latent action alignment from training-only rendered semantic priors, draft image features, and supervised halting signals, followed by Latent-Flow GRPO to align the latent-to-image rollout with terminal visual feedback. This provides a control path from inferred relations, bindings, and knowledge cues to the generation process. Instantiated on BAGEL-7B-MoT, LAC consistently improves compositional and knowledge-grounded generation across GenEval, WISE, and T2I-CompBench, with the largest gains on spatial relations, attribute binding, and world-knowledge-sensitive prompts. Ablations and latent interventions show that the learned action trajectory is consumed by the generator, suggesting that unified generation benefits when understanding is not only encoded, but made actionable during generation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 49 references that are in the map. First posted on arXiv 16 May 2026.
Builds on
46 of 49 references.
References · 46
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
34 more
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
- 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
- 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 256
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
- 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
- 2025Latent Visual Reasoningcited by 104
- 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 90
- 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 82
- 2025ImageGen-CoT: Enhancing Text-to-Image In-context Learning with Chain-of-Thought Reasoningcited by 37
- 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 31
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 28
- 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 26
- 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 25
- 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 20
- 2026Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generationcited by 20
- 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generationcited by 18
- 2026LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoningcited by 15
- 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationcited by 13
- 2025Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generationcited by 11
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 10
- 2025Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Modelscited by 10
- 2026ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thoughtcited by 10
- 2025MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoningcited by 8
- 2026Show, Don't Tell: Morphing Latent Reasoning into Image Generationcited by 8
- 2025Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance for Text-to-Image Generationcited by 5
- 2026Multimodal Latent Reasoning via Hierarchical Visual Cues Injectioncited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning
- 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning
- 2026Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models
- 2026Being-H0.7: A Latent World-Action Model from Egocentric Videos
- 2026From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models
- 2026PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
- 2026Towards Predictive, Aligned, and Scalable Robot Learning
- 2026Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning