# CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward

arXiv: https://arxiv.org/abs/2504.03724 · HTML: https://mlsanity.com/paper/2504.03724/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [Visual Classification via Description from Large Language Models](https://mlsanity.com/paper/2210.07183/) (2022)
- [R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization](https://mlsanity.com/paper/2503.12937/) (2025)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning](https://mlsanity.com/paper/2502.19634/) (2025)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models](https://mlsanity.com/paper/2212.10846/) (2022)
- [Switching Convolutional Neural Network for Crowd Counting](https://mlsanity.com/paper/1708.00199/) (2017)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning](https://mlsanity.com/paper/2503.18013/) (2025)
- [A Low-Shot Object Counting Network With Iterative Prototype Adaptation](https://mlsanity.com/paper/2211.08217/) (2022)
- [Few-shot Object Counting with Similarity-Aware Feature Enhancement](https://mlsanity.com/paper/2201.08959/) (2022)
- [CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model](https://mlsanity.com/paper/2304.04231/) (2023)
- [VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting](https://mlsanity.com/paper/2312.16580/) (2023)
- [Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methods](https://mlsanity.com/paper/2005.02162/) (2020)
- [CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting](https://mlsanity.com/paper/1707.09605/) (2017)
- [Open-world Text-specified Object Counting](https://mlsanity.com/paper/2306.01851/) (2023)
- [T-Rex: Counting by Visual Prompting](https://mlsanity.com/paper/2311.13596/) (2023)
- [Benchmarking Large Language Models for Image Classification of Marine Mammals](https://mlsanity.com/paper/2410.19848/) (2024)
- [Mask Focal Loss: A unifying framework for dense crowd counting with canonical object detection networks](https://mlsanity.com/paper/2212.11542/) (2022)
- [Counting Manatee Aggregations using Deep Neural Networks and Anisotropic Gaussian Kernel](https://mlsanity.com/paper/2311.02315/) (2023)

## Built on by (most cited first, 15 of 15)

- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2506.07905/) (2025)
- [Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewards](https://mlsanity.com/paper/2505.02686/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [ReDit: Reward Dithering for Improved LLM Policy Optimization](https://mlsanity.com/paper/2506.18631/) (2025)
- [What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis](https://mlsanity.com/paper/2602.12395/) (2026)
- [Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design](https://mlsanity.com/paper/2603.00152/) (2026)
- [SITUATE -- Synthetic Object Counting Dataset for VLM training](https://mlsanity.com/paper/2602.00108/) (2026)
- [HOI-R1: Exploring the Potential of Multimodal Large Language Models for Human-Object Interaction Detection](https://mlsanity.com/paper/2510.05609/) (2025)
- [Structure Over Scale: Learning Visual Reasoning from Pedagogical Video](https://mlsanity.com/paper/2601.23251/) (2026)
- [REC-RL: Referring expression counting via Gaussian and range-based reward optimization](https://mlsanity.com/paper/2605.16460/) (2026)
- [Cross-Resolution SAR Target Detection Using Structural Hierarchy Adaptation and Reliable Adjacency Alignment](https://mlsanity.com/paper/2507.08290/) (2025)
- [When Large Language Models Do Not Work: Online Incivility Prediction through Graph Neural Networks](https://mlsanity.com/paper/2512.07684/) (2025)
- [CrowdCue: Specialist-Cue Conditioning for Vision-Language Crowd Counting](https://mlsanity.com/paper/2609.23012/) (2026)

## Abstract (arXiv)

We propose Fuzzy Group Relative Policy Reward (FGRPR), a novel framework that integrates Group Relative Policy Optimization (GRPO) with a fuzzy reward function to enhance learning efficiency. Unlike the conventional binary 0/1 accuracy reward, our fuzzy reward model provides nuanced incentives, encouraging more precise outputs. Experimental results demonstrate that GRPO with a standard 0/1 accuracy reward underperforms compared to supervised fine-tuning (SFT). In contrast, FGRPR, applied to Qwen2.5-VL(3B and 7B), surpasses all baseline models, including GPT4o, LLaMA2(90B), and SFT, across five in-domain datasets. On an out-of-domain dataset, FGRPR achieves performance comparable to SFT but excels when target values are larger, as its fuzzy reward function assigns higher rewards to closer approximations. This approach is broadly applicable to tasks where the precision of the answer is critical. Code and data: https://github.com/yeyimilk/CrowdVLM-R1