mlsanityData to 14 Oct 2026

Robust Object Detection for Autonomous Driving via Curriculum-Guided Group Relative Policy Optimization

arXiv 2509.22688 ↗First posted 19 Sep 2025cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) excel in vision-language reasoning but often struggle with structured perception tasks requiring precise localization and robustness. We propose a reinforcement learning framework that augments Group Relative Policy Optimization (GRPO) with curriculum-based data scheduling and difficulty-aware filtering. This approach stabilizes optimization under sparse, noisy rewards and enables progressive adaptation to complex samples. Evaluations on autonomous driving benchmarks demonstrate substantial improvements in detection accuracy and robustness. Ablation studies confirm the importance of reward design, KL regularization, and curriculum pacing for convergence stability and generalization. Our findings highlight reinforcement-driven optimization with structured data curricula as a scalable path toward robust and interpretable multimodal detection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 37 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

29 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  12. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
16 more
  1. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  2. 2016Modeling Context in Referring Expressionscited by 1,063
  3. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  4. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  5. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  6. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  7. 2021Curriculum Learning: A Surveycited by 259
  8. 2017Automated Curriculum Learning for Neural Networkscited by 257
  9. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  10. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  11. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
  12. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  13. 2019Augmentation for small object detectioncited by 73
  14. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 49
  15. 2025AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPOcited by 33
  16. 2025AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detectioncited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving
  2. 2025DriveMind: A Dual Visual Language Model-based Reinforcement Learning Framework for Autonomous Driving
  3. 2026OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
  4. 2026GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
  5. 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving
  6. 2026Structured Role-Aware Policy Optimization for Multimodal Reasoning
  7. 2026S-GRPO: Unified Post-Training for Large Vision-Language Models
  8. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning