mlsanityData to 7 Oct 2026

Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification

arXiv 2609.07148 ↗First posted 7 Sep 2026cs.LG cs.CVcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

While Reinforcement Learning (RL) effectively incentivizes reasoning in Large Language Models, current pipelines are hindered by training instability and rapid entropy collapse. These limitations often stem from "Rollout Silencing" and low-quality gradient signals in standard sampling procedures. In this work, we propose a robust, data-centric framework to stabilize RL training. We first introduce Potential-Aware Query Mining (PAQM), which filters data dynamically to focus on the "Distillation Zone"---samples with high potential for capability elicitation. Furthermore, we present Hybrid Stratified Replay (HSR), a novel mechanism that restructures batches by stratifying rollouts based on Path Entropy, a rollout-level confidence proxy, and outcome reward. Within each optimization step, HSR reuses current-policy "Stability Anchors" and "Hard Negatives" to construct high-contrast optimization groups, then clears its buffers before the next step. This approach mitigates entropy collapse while improving the utilization of learning signals under limited compute. Our method outperforms strong baselines on complex reasoning tasks, offering a principled solution for stable and efficient RL fine-tuning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 59 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

44 of 59 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards25 Dec
  2. 2025M-GRPO: Stabilizing Self-Supervised Reinforcement Learning for Large Language Models with Momentum-Anchored Policy Optimization15 Dec
  3. 2025Shrinking the Variance: Shrinkage Baselines for Reinforcement Learning with Verifiable Rewards5 Nov
  4. 2025Sharpness-Guided Group Relative Policy Optimization via Probability Shaping29 Oct
  5. 2025RoRecomp: Enhancing Reasoning Efficiency via Rollout Response Recomposition in Reinforcement Learning30 Sep
  6. 2025Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning26 Sep
  7. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback3 Jun
  8. 2025Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning25 Apr
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret3 Mar
  11. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
2 more
  1. 2023Let's Verify Step by Step31 May
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  7. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  8. 2025Seed1.5-VL Technical Reportcited by 367
  9. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
  10. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  11. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  12. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 240
18 more
  1. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  2. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  4. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 143
  5. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  6. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 51
  7. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resourcescited by 37
  8. 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 33
  9. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 30
  10. 2026Gen-Searcher: Reinforcing Agentic Search for Image Generationcited by 25
  11. 2025Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shufflecited by 19
  12. 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 18
  13. 2025SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoringcited by 17
  14. 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesiscited by 16
  15. 2025Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curationcited by 13
  16. 2025EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Controlcited by 13
  17. 2025Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrievalcited by 12
  18. 2026Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agentcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
  2. 2025Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs
  3. 2025Stable and Efficient Single-Rollout RL for Multimodal Reasoning
  4. 2025CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention
  5. 2025Revisiting Entropy in Reinforcement Learning for Large Reasoning Models
  6. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
  7. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  8. 2026STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability