mlsanityData to 14 Oct 2026

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

arXiv 2505.13031 ↗First posted 19 May 2025cs.AIcited by 30

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent text-to-image systems face limitations in handling multimodal inputs and complex reasoning tasks. We introduce MindOmni, a unified multimodal large language model that addresses these challenges by incorporating reasoning generation through reinforcement learning. MindOmni leverages a three-phase training strategy: i) design of a unified vision language model with a decoder-only diffusion module, ii) supervised fine-tuning with Chain-of-Thought (CoT) instruction data, and iii) our proposed Reasoning Generation Policy Optimization (RGPO) algorithm, utilizing multimodal feedback to effectively guide policy updates. Experimental results demonstrate that MindOmni outperforms existing models, achieving impressive performance on both understanding and generation benchmarks, meanwhile showcasing advanced fine-grained reasoning generation capabilities, especially with mathematical reasoning instruction. All codes will be made public at https://github.com/TencentARC/MindOmni

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 85 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

49 of 85 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
  2. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  3. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
  4. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step23 Jan
  5. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 43

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
31 more
  1. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  2. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  4. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  5. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  6. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  7. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  8. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  9. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  10. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  11. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  12. 2024OmniGen: Unified Image Generationcited by 394
  13. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  14. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  15. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  16. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  17. 2025Transfer between Modalities with MetaQueriescited by 236
  18. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  19. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  20. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  21. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  22. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  23. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  24. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  25. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  26. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 86
  27. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 59
  28. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 54
  29. 2025TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generationcited by 51
  30. 2024Prism: A Framework for Decoupling and Assessing the Capabilities of VLMscited by 32
  31. 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Modelcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 17.

30 papers.

Sort:

By area:

  1. 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 41
  2. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 37
  3. 2025SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learningcited by 31
  4. 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
  5. 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
  6. 2026Demystifying Video Reasoningcited by 14
  7. 2025HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generationcited by 13
  8. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editingcited by 12
  9. 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 12
  10. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 10
  11. 2026SpatialEdit: Benchmarking Fine-Grained Image Spatial Editingcited by 10
  12. 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPOcited by 9
18 more
  1. 2025COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligencecited by 9
  2. 2025MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoningcited by 8
  3. 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scalingcited by 8
  4. 2026PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generationcited by 8
  5. 2026M3: High-fidelity Text-to-Image Generation via Multi-Modal, Multi-Agent and Multi-Round Visual Reasoningcited by 6
  6. 2025LoRA-Gen: Specializing Large Language Model via Online LoRA Generationcited by 5
  7. 2025MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Promptscited by 5
  8. 2025ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinementcited by 5
  9. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generationcited by 3
  10. 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editingcited by 3
  11. 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 3
  12. 2026EffectMaker: Unifying Reasoning and Generation for Customized Visual Effect Creationcited by 2
  13. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 1
  14. 2026Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinkingcited by 0
  15. 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentationcited by 0
  16. 2026Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learningcited by 0
  17. 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generationcited by 0
  18. 2026Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  2. 2026Balanced Thinking: Improving Chain of Thought Training in Vision Language Models
  3. 2026FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation
  4. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models
  5. 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
  6. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
  7. 2026Reasoning with Image Generation
  8. 2025EasyARC: Evaluating Vision Language Models on True Visual Reasoning