mlsanityData to 14 Oct 2026

ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Model

arXiv 2510.24285 ↗First posted 28 Oct 2025cs.CV cs.AI cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The limited capacity for fine-grained visual perception presents a critical bottleneck for Vision-Language Models (VLMs) in real-world applications. Addressing this is challenging due to the scarcity of high-quality data and the limitations of existing methods: supervised fine-tuning (SFT) often compromises general capabilities, while reinforcement fine-tuning (RFT) prioritizes textual reasoning over visual perception. To bridge this gap, we propose a novel two-stage task that structures visual perception learning as a coarse-to-fine progressive process. Based on this task formulation, we develop ViPER, a self-bootstrapping framework specifically designed to enable iterative evolution through self-critiquing and self-prediction. By synergistically integrating image-level and instance-level reconstruction with a two-stage reinforcement learning strategy, ViPER establishes a closed-loop training paradigm, where internally synthesized data directly fuel the enhancement of perceptual ability. Applied to the Qwen2.5-VL family, ViPER produces the Qwen-Viper series. With an average gain of 1.7% on seven comprehensive benchmarks spanning various tasks and up to 6.0% on fine-grained perception, Qwen-Viper consistently demonstrates superior performance across different vision-language scenarios while maintaining generalizability. Beyond enabling self-improvement in perceptual capabilities, ViPER provides concrete evidence for the reciprocal relationship between generation and understanding, a breakthrough to developing more autonomous and capable VLMs.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 60 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

53 of 60 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  3. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  4. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  5. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  6. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  7. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning27 Nov

Other references · 46

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
34 more
  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  4. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  6. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  7. 2025Qwen-Image Technical Reportcited by 1,044
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  9. 2023A Survey on Multimodal Large Language Modelscited by 909
  10. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  11. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  12. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  13. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  14. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  15. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  16. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  17. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  18. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
  19. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  20. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
  21. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 103
  22. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
  23. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  24. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  25. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 65
  26. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
  27. 2023Language Models as Black-Box Optimizers for Vision-Language Modelscited by 36
  28. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classificationcited by 35
  29. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
  30. 2024Active Data Curation Effectively Distills Large-Scale Multimodal Modelscited by 19
  31. 2025Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chainscited by 12
  32. 2025HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMscited by 3
  33. 2025Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learningcited by 3
  34. 2025AME: Aligned Manifold Entropy for Robust Vision-Language Distillationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025VIPER: Visual Perception and Explainable Reasoning for Sequential Decision-Making
  2. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
  3. 2026Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains
  4. 2025Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective
  5. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  6. 2025VisPlay: Self-Evolving Vision-Language Models from Images
  7. 2025VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
  8. 2026TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models