mlsanityData to 7 Oct 2026

VisPlay: Self-Evolving Vision-Language Models from Images

arXiv 2511.15661 ↗First posted 19 Nov 2025cs.CV cs.AI cs.CLcited by 42

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Reinforcement learning (RL) provides a principled framework for improving Vision-Language Models (VLMs) on complex reasoning tasks. However, existing RL approaches often rely on human-annotated labels or task-specific heuristics to define verifiable rewards, both of which are costly and difficult to scale. We introduce VisPlay, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning abilities using large amounts of unlabeled image data. Starting from a single base VLM, VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses. These roles are jointly trained with Group Relative Policy Optimization (GRPO), which incorporates diversity and difficulty rewards to balance the complexity of generated questions with the quality of the silver answers. VisPlay scales efficiently across two model families. When trained on Qwen2.5-VL and MiMo-VL, VisPlay achieves consistent improvements in visual reasoning, compositional generalization, and hallucination reduction across eight benchmarks, including MM-Vet and MMMU, demonstrating a scalable path toward self-evolving multimodal intelligence. The project page is available at https://bruno686.github.io/VisPlay/

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 50 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

39 of 50 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning1 Oct
  2. 2025Self-Rewarding Vision-Language Model via Reasoning Decomposition27 Aug
  3. 2025SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning22 Jul
  4. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning20 May
  5. 2025Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning20 May
  6. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  7. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  8. 2024Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training22 Apr
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb

Other references · 30

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  7. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  8. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  9. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  10. 2025Reasoning with Exploration: An Entropy Perspectivecited by 277
  11. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  12. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 134
18 more
  1. 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 84
  2. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  3. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  4. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 59
  5. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 58
  6. 2025One Token to Fool LLM-as-a-Judgecited by 56
  7. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  8. 2025Language Self-Play For Data-Free Trainingcited by 42
  9. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  10. 2024Weak-to-Strong Reasoningcited by 32
  11. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  12. 2024VisionLLaMA: A Unified LLaMA Backbone for Vision Taskscited by 28
  13. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 28
  14. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 27
  15. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoningcited by 24
  16. 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolutioncited by 24
  17. 2025Deep Self-Evolving Reasoningcited by 3
  18. 2025Self-Evolving Visual Concept Library using Vision-Language Criticscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 39.

42 papers.

Sort:

By area:

  1. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  2. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 28
  3. 2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datacited by 16
  4. 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolvingcited by 15
  5. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 15
  6. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 13
  7. 2026Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoningcited by 10
  8. 2026Visually-Guided Policy Optimization for Multimodal Reasoningcited by 10
  9. 2026R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Trainingcited by 9
  10. 2026When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoningcited by 8
  11. 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 8
  12. 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolutioncited by 7
30 more
  1. 2026Active Zero: Self-Evolving Vision-Language Models through Active Environment Explorationcited by 7
  2. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 7
  3. 2026iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Modelscited by 6
  4. 2026Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Modelscited by 5
  5. 2026Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Taskscited by 4
  6. 2026SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environmentscited by 4
  7. 2026CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphscited by 3
  8. 2026Adaptive Time Series Reasoning via Segment Selectioncited by 3
  9. 2026Stabilizing Unsupervised Self-Evolution of MLLMs via Continuous Softened Retracing reSamplingcited by 3
  10. 2026EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformationscited by 3
  11. 2026EvoGround: Self-Evolving Video Agents for Video Temporal Groundingcited by 3
  12. 2026EvoVid: Temporal-Centric Self-Evolution for Video Large Language Modelscited by 2
  13. 2026Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spotscited by 2
  14. 2026AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedbackcited by 1
  15. 2026TTSR: Test-Time Self-Evolving via Reflectioncited by 1
  16. 2026Discovering Failure Modes in Vision-Language Models using RLcited by 1
  17. 2026Structured Role-Aware Policy Optimization for Multimodal Reasoningcited by 1
  18. 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Datacited by 1
  19. 2026RISE: Reliable Improvement in Self-Evolving Vision-Language Modelscited by 1
  20. 2026DUEL: Adversarial Self-Play for Multimodal Reasoningcited by 1
  21. 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
  22. 2026Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoningcited by 1
  23. 2025Probing the effectiveness of World Models for Spatial Reasoning through Test-time Scalingcited by 0
  24. 2026From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Modelscited by 0
  25. 2026Dual-Imbalance Continual Learning for Real-World Food Recognitioncited by 0
  26. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutionscited by 0
  27. 2026Seirenes: Adversarial Self-Play with Evolving Distractions for LLM Reasoningcited by 0
  28. 2026Teach-to-Reason: Competition-Guided Reasoning with a Self-Improving Teachercited by 0
  29. 2026Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Designcited by 0
  30. 2026VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Questioning Vision-Language Models: Reinforcement Learning for Compositional Visual Reasoning
  2. 2026Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
  3. 2026V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation
  4. 2026Trinity: Self-Evolving Vision-Language Models with a Self-Verifier
  5. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  6. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  7. 2026TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models
  8. 2026Self-Evolving Visual Questioner