Scaling Vision-Language Reward Learning for Robot Manipulation in Parallel Simulation
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models also fits: Prompt learning and adaptation of CLIP
Abstract (arXiv)
Vision-language models (VLMs) can replace human annotators in preference-based reward learning, but sequential API requests and single-environment data collection make training slow and costly. We present RAPID (Reward learning with Adaptive Parallel Image Diversity), a system that couples GPU-parallel rollout with data-aware policy updates, single-request preference labeling, automatic reward stabilization, and representative image sampling. We evaluate these components on five Franka Panda manipulation tasks in IsaacLab. Parallel rollout and adaptive updates provide the first substantial reduction in training time: under matched two-stage prompting, mean runtime falls from 9.18 to 3.13 hours. With all RAPID components enabled, training completes in 1.15 hours using 896 rather than 19,840 API calls per run, and aggregate final success rises from 86.3% to 98.7%. This represents an 8.0× end-to-end speedup and a 95.5% reduction in API usage. An offline evaluation with Gemma~3 12B and GPT-4.1 mini demonstrates that single-request prompting reduces labeling latency and cost across both models. Code is available at: https://github.com/rapid-vlm/rapid-vlm-rl.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 18 Sep 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
- 2026Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models
- 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
- 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
- 2026Learning to Accelerate Vision-Language-Action Models through Adaptive Visual Token Caching
- 2026Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models
- 2026RMTL: Reinforced Micro-task Learning for Long-Horizon Manipulation with VLM Rewards ↗
- 2026MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Models