mlsanityData to 7 Oct 2026

Visual-RFT: Visual Reinforcement Fine-Tuning

arXiv 2503.01785 ↗First posted 3 Mar 2025cs.CVcited by 501

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Reinforcement Fine-Tuning (RFT) in Large Reasoning Models like OpenAI o1 learns from feedback on its answers, which is especially useful in applications when fine-tuning data is scarce. Recent open-source work like DeepSeek-R1 demonstrates that reinforcement learning with verifiable reward is one key direction in reproducing o1. While the R1-style model has demonstrated success in language models, its application in multi-modal domains remains under-explored. This work introduces Visual Reinforcement Fine-Tuning (Visual-RFT), which further extends the application areas of RFT on visual tasks. Specifically, Visual-RFT first uses Large Vision-Language Models (LVLMs) to generate multiple responses containing reasoning tokens and final answers for each input, and then uses our proposed visual perception verifiable reward functions to update the model via the policy optimization algorithm such as Group Relative Policy Optimization (GRPO). We design different verifiable reward functions for different perception tasks, such as the Intersection over Union (IoU) reward for object detection. Experimental results on fine-grained image classification, few-shot object detection, reasoning grounding, as well as open-vocabulary object detection benchmarks show the competitive performance and advanced generalization ability of Visual-RFT compared with Supervised Fine-tuning (SFT). For example, Visual-RFT improves accuracy by 24.3% over the baseline in one-shot fine-grained image classification with around 100 samples. In few-shot object detection, Visual-RFT also exceeds the baseline by 21.9 on COCO's two-shot setting and 15.4 on LVIS. Our Visual-RFT represents a paradigm shift in fine-tuning LVLMs, offering a data-efficient, reward-driven approach that enhances reasoning and adaptability for domain-specific tasks.

Key facts. Cited by 501 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 58 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

46 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model21 Jan
  2. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug

Other references · 44

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Segment Anythingcited by 8,158
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
32 more
  1. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  4. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  7. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  8. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  9. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  10. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  11. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  12. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  13. 2024InternLM2 Technical Reportcited by 391
  14. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  15. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  16. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  17. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  18. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  19. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  20. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  21. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  22. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
  23. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
  24. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
  25. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  26. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  27. 2024o1-Coder: an o1 Replication for Codingcited by 77
  28. 2024Preference Optimization for Reasoning with Pseudo Feedbackcited by 38
  29. 2024MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Modelscited by 34
  30. 2024CodeDPO: Aligning Code Models with Self Generated and Verified Source Codecited by 32
  31. 2023Detect Everything with Few Examplescited by 13
  32. 2024SIA-OVD: Shape-Invariant Adapter for Bridging the Image-Region Gap in Open-Vocabulary Detectioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 324; 2026: 177.

501 papers.

Sort:

By area:

  1. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  5. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  6. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  7. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  8. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  9. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  10. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
  11. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
  12. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
38 more
  1. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
  2. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  4. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  5. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 116
  6. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  7. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  8. 2025Latent Visual Reasoningcited by 99
  9. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  10. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  11. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
  12. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  13. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  14. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  15. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  16. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
  17. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 86
  18. 2025Visual Planning: Let's Think Only with Imagescited by 85
  19. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  20. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  21. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  22. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  23. 2025VGR: Visual Grounded Reasoningcited by 70
  24. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learningcited by 68
  25. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  26. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
  27. 2025R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learningcited by 65
  28. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 63
  29. 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 59
  30. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  31. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  32. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  33. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53
  34. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  35. 2025VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learningcited by 50
  36. 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
  37. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  38. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46

50 most cited of 501 All 501 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
  2. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
  3. 2025VIDEOP2R: Video Understanding from Perception to Reasoning
  4. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  5. 2025Reinforced Visual Perception with Tools
  6. 2026Visually-Guided Policy Optimization for Multimodal Reasoning
  7. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning
  8. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning