mlsanityData to 7 Oct 2026

MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Models

arXiv 2602.15872 ↗First posted 28 Jan 2026cs.RO cs.CV cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Designing dense reward functions is pivotal for efficient robotic Reinforcement Learning (RL). However, most dense rewards rely on manual engineering, which fundamentally limits the scalability and automation of reinforcement learning. While Vision-Language Models (VLMs) offer a promising path to reward design, naive VLM rewards often misalign with task progress, struggle with spatial grounding, and show limited understanding of task semantics. To address these issues, we propose MARVL-Multi-stAge guidance for Robotic manipulation via Vision-Language models. MARVL fine-tunes a VLM for spatial and semantic consistency and decomposes tasks into multi-stage subtasks with task direction projection for trajectory sensitivity. Empirically, MARVL significantly outperforms existing VLM-reward methods on the Meta-World benchmark, demonstrating superior sample efficiency and robustness on sparse-reward manipulation tasks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 52 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

21 of 52 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Real-World Offline Reinforcement Learning from Vision Language Model Feedback8 Nov
  2. 2024The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards24 Sep
  3. 2024RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation27 Jun
  4. 2024Code as Reward: Empowering Reinforcement Learning with VLMs7 Feb
  5. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback6 Feb
  6. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learning5 Feb
  7. 2023Vision-Language Models as a Source of Rewards14 Dec
  8. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning19 Oct

Other references · 13

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  4. 2017Deep Reinforcement Learning: An Overviewcited by 327
  5. 2023Vision-Language Models as Success Detectorscited by 138
  6. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 56
  7. 2026RoboReward: General-Purpose Vision-Language Reward Models for Roboticscited by 46
  8. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  9. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  10. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  11. 2023Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectoriescited by 9
  12. 2023CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observationscited by 8
1 more
  1. 2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
  2. 2026CounterAlign: Counterfactual Supervision for Vision-Language-Action Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
  2. 2025Training-free Generation of Temporally Consistent Rewards from VLMs
  3. 2024Affordance-Guided Reinforcement Learning via Visual Prompting
  4. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models
  5. 2026RMTL: Reinforced Micro-task Learning for Long-Horizon Manipulation with VLM Rewards ↗
  6. 2024Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance
  7. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
  8. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey