mlsanityData to 14 Oct 2026

One RL to See Them All: Visual Triple Unified Reinforcement Learning

arXiv 2505.18129 ↗First posted 23 May 2025cs.CV cs.CLcited by 30

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) is becoming an important direction for post-training vision-language models (VLMs), but public training methodologies for unified multimodal RL remain much less mature, especially for heterogeneous reasoning and perception-heavy tasks. We propose V-Triune, a Visual Triple Unified Reinforcement Learning methodology for unified multimodal RL. It organizes training around three coordinated abstractions: Sample-Level Reward Routing, Verifier-Level Outcome Verification, and Source-Level Diagnostics. Within this methodology, Dynamic IoU provides localization-specific reward shaping that avoids reward ambiguity under loose thresholds and reward sparsity under strict ones. Built on V-Triune, we develop Orsta (7B, 32B), a family of models jointly trained on eight reasoning and perception tasks. Under matched budgets, unified training matches or outperforms specialist mixtures. The final Orsta models improve over their backbones on MEGA-Bench, compare favorably with strong multi-task RL-VLM baselines, and transfer these gains to a broad set of downstream benchmarks. These results show that unified RL can improve both reasoning and perception within a single VLM RL pipeline.The V-Triune system, along with the Orsta models, is publicly available at https://github.com/MiniMax-AI/One-RL-to-See-Them-All.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 57 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

37 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  2. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  3. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  4. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme3 Apr
  5. 2025OThink-MR1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning20 Mar
  6. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar

Other references · 31

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  10. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  11. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  12. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
19 more
  1. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  2. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  3. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 303
  4. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  5. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  6. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  7. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 161
  8. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  9. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 120
  10. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 108
  11. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  12. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  13. 2024PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patternscited by 57
  14. 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledgecited by 55
  15. 2024MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Taskscited by 41
  16. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  17. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classificationcited by 35
  18. 2024Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoningcited by 25
  19. 2023How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detectioncited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 16.

30 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
  3. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  4. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  5. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agentscited by 38
  6. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
  7. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  8. 2025Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shufflecited by 19
  9. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  10. 2026Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thoughtcited by 9
  11. 2025Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoningcited by 7
  12. 2026Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Modelscited by 7
18 more
  1. 2025Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Startcited by 5
  2. 2026Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimizationcited by 5
  3. 2026Large Vision-Language Models Get Lost in Attentioncited by 5
  4. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshapingcited by 5
  5. 2025Visual Programmability: A Guide for Code-as-Thought in Chart Understandingcited by 4
  6. 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligencecited by 4
  7. 2026Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalizationcited by 4
  8. 2026Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimizationcited by 2
  9. 2025PairUni: Pairwise Training for Unified Multimodal Language Modelscited by 1
  10. 2026Video-MOPD: Multi-Teacher On-Policy Distillation for Video Understandingcited by 1
  11. 2026Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Driftcited by 1
  12. 2025Revisiting the Data Sampling in Multimodal Post-training from a Difficulty-Distinguish Viewcited by 0
  13. 2026Weather-R1: Logically Consistent Reinforcement Fine-Tuning for Multimodal Reasoning in Meteorologycited by 0
  14. 2026ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoningcited by 0
  15. 2026CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Modelscited by 0
  16. 2026MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Groundingcited by 0
  17. 2026EviRover: Reinforcing Agentic Perception Beyond a Glancecited by 0
  18. 2026GeoPID: Decomposing and Steering Visual Information in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  2. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models
  3. 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis
  4. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
  5. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
  6. 2026SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
  7. 2026IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
  8. 2026VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning