mlsanityData to 13 Oct 2026

From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Grounding

arXiv 2605.15951 ↗First posted 15 May 2026cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Finetuning Large Vision-Language Models with reinforcement learning has emerged as a promising approach to enhance their capability in object-level grounding. However, existing methods, mainly based on GRPO, assign rewards at the response level. Such sparse reward, often criterion-induced, leads to minimal learning signals when all candidate responses fail in challenging scenarios. In this work, we propose a group-revision optimisation paradigm that enhances learning on hard cases. It begins with a sampled initial response and generates a set of revised candidates to explore improved grounding outcomes. Inspired by reward shaping, we introduce a consolidation process that quantifies each candidate's improvement over the initial attempt and converts it into informative shaping signals. These signals are used to both refine the reward and modulate the advantage, amplifying the influence of high-quality revisions. Our method achieves consistent gains across referring and reasoning segmentation, REC, and counting benchmarks compared with prior GRPO-based models. Our code is available at https://github.com/yyliu01/GroupRevision.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 87 of its 100 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

87 of 100 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoning2 Jul
  2. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoning13 Mar
  3. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Models4 Dec
  4. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos29 Sep
  5. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models25 Sep
  6. 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models19 Apr
  7. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model19 Dec
  8. 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
  9. 2023GLaMM: Pixel Grounding Large Multimodal Model6 Nov
  10. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
  11. 2023Vision-Language Models for Vision Tasks: A Survey3 Apr
  12. 2023Teaching CLIP to Count to Ten23 Feb
2 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 73

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2023Segment Anythingcited by 8,195
  3. 2023Visual Instruction Tuningcited by 7,827
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,103
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,831
  6. 2025Qwen2.5-VL Technical Reportcited by 5,499
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,191
  8. 2017Deep reinforcement learning from human preferencescited by 3,871
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
  10. 2023Let's Verify Step by Stepcited by 3,672
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,007
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,820
38 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,703
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,692
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,068
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,052
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,003
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,798
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,709
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,637
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,586
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,461
  11. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  12. 2017Hindsight Experience Replaycited by 1,104
  13. 2016Modeling Context in Referring Expressionscited by 1,057
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 799
  15. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 677
  16. 2018Object Hallucination in Image Captioningcited by 663
  17. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 520
  18. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 506
  19. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 502
  20. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 470
  21. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 466
  22. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 407
  23. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 384
  24. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  25. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 348
  26. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  27. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 239
  28. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 237
  29. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 227
  30. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 173
  31. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 144
  32. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  33. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 123
  34. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 84
  35. 2024Process Reward Model with Q-Value Rankingscited by 79
  36. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 75
  37. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
  38. 2023PerceptionGPT: Effectively Fusing Visual Perception into LLMcited by 59

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perceptioncited by 1
  2. 2026Revisit to Segment: Working Memory Distillation for Reasoning Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Grounding for Object-Level Generalization in Reinforcement Learning
  2. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning
  3. 2026Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning
  4. 2026Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models
  5. 2025KARL: Knowledge-Aware Reasoning and Reinforcement Learning for Knowledge-Intensive Visual Grounding
  6. 2025High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning
  7. 2024ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling
  8. 2024Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves?