ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Model
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
The limited capacity for fine-grained visual perception presents a critical bottleneck for Vision-Language Models (VLMs) in real-world applications. Addressing this is challenging due to the scarcity of high-quality data and the limitations of existing methods: supervised fine-tuning (SFT) often compromises general capabilities, while reinforcement fine-tuning (RFT) prioritizes textual reasoning over visual perception. To bridge this gap, we propose a novel two-stage task that structures visual perception learning as a coarse-to-fine progressive process. Based on this task formulation, we develop ViPER, a self-bootstrapping framework specifically designed to enable iterative evolution through self-critiquing and self-prediction. By synergistically integrating image-level and instance-level reconstruction with a two-stage reinforcement learning strategy, ViPER establishes a closed-loop training paradigm, where internally synthesized data directly fuel the enhancement of perceptual ability. Applied to the Qwen2.5-VL family, ViPER produces the Qwen-Viper series. With an average gain of 1.7% on seven comprehensive benchmarks spanning various tasks and up to 6.0% on fine-grained perception, Qwen-Viper consistently demonstrates superior performance across different vision-language scenarios while maintaining generalizability. Beyond enabling self-improvement in perceptual capabilities, ViPER provides concrete evidence for the reciprocal relationship between generation and understanding, a breakthrough to developing more autonomous and capable VLMs.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 60 references that are in the map. First posted on arXiv 28 Oct 2025.
Builds on
53 of 60 references.
Closest ancestors · 7
Same direction, newest first
- 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning27 Nov
Other references · 46
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
34 more
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2025Qwen-Image Technical Reportcited by 1,044
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2023A Survey on Multimodal Large Language Modelscited by 909
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
- 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
- 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
- 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
- 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 103
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
- 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
- 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
- 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 65
- 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
- 2023Language Models as Black-Box Optimizers for Vision-Language Modelscited by 36
- 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classificationcited by 35
- 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
- 2024Active Data Curation Effectively Distills Large-Scale Multimodal Modelscited by 19
- 2025Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chainscited by 12
- 2025HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMscited by 3
- 2025Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learningcited by 3
- 2025AME: Aligned Manifold Entropy for Robust Vision-Language Distillationcited by 2
Built on by
Citing papers per year: 2025: 3; 2026: 8.
11 papers.
- 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
- 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 27
- 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolvingcited by 19
- 2026Forest Before Trees: Latent Superposition for Efficient Visual Reasoningcited by 18
- 2025Training Multi-Image Vision Agents via End2End Reinforcement Learningcited by 4
- 2026EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformationscited by 4
- 2026Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energycited by 3
- 2026Evidence-RL: Towards Evidence-intensive Visual Reasoningcited by 3
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1
- 2026Allocate Before You Embed: Adaptive Visual Input Allocation for Video Embeddingscited by 1
- 2026VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoningcited by 0
- 2026VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoning7 Oct 2026
- 2026Allocate Before You Embed: Adaptive Visual Input Allocation for Video Embeddings1 Sep 2026
- 2026Evidence-RL: Towards Evidence-intensive Visual Reasoning8 Aug 2026
- 2026EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations20 Apr 2026
- 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering9 Mar 2026
- 2026Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energy6 Mar 2026
- 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolving30 Jan 2026
- 2026Forest Before Trees: Latent Superposition for Efficient Visual Reasoning11 Jan 2026
- 2025Training Multi-Image Vision Agents via End2End Reinforcement Learning5 Dec 2025
- 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning25 Nov 2025
- 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards20 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025VIPER: Visual Perception and Explainable Reasoning for Sequential Decision-Making
- 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
- 2026Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains
- 2025Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective
- 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
- 2025VisPlay: Self-Evolving Vision-Language Models from Images
- 2025VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
- 2026TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models