mlsanityData to 7 Oct 2026

Vision-Language Models as Success Detectors

arXiv 2303.07280 ↗First posted 13 Mar 2023cs.CV cs.AI cs.LGcited by 138

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

also in 1 more
Abstract (arXiv)

Detecting successful behaviour is crucial for training intelligent agents. As such, generalisable reward models are a prerequisite for agents that can learn to generalise their behaviour. In this work we focus on developing robust success detectors that leverage large, pretrained vision-language models (Flamingo, Alayrac et al. (2022)) and human reward annotations. Concretely, we treat success detection as a visual question answering (VQA) problem, denoted SuccessVQA. We study success detection across three vastly different domains: (i) interactive language-conditioned agents in a simulated household, (ii) real world robotic manipulation, and (iii) "in-the-wild" human egocentric videos. We investigate the generalisation properties of a Flamingo-based success detection model across unseen language and visual changes in the first two domains, and find that the proposed method is able to outperform bespoke reward models in out-of-distribution test scenarios with either variation. In the last domain of "in-the-wild" human videos, we show that success detection on unseen real videos presents an even more challenging generalisation task warranting future work. We hope our initial results encourage further work in real world success detection and reward modelling.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 56 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

43 of 56 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
31 more
  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  5. 2022A Generalist Agentcited by 802
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  9. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  10. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  11. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  12. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  13. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  14. 2022Teaching language models to support answers with verified quotescited by 290
  15. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
  16. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  17. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  18. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  19. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  20. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  21. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  22. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
  23. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  24. 2022Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillationcited by 70
  25. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  26. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  27. 2020Imitating Interactive Intelligencecited by 55
  28. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  29. 2021Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learningcited by 40
  30. 2022Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedbackcited by 26
  31. 2022Evaluating Multimodal Interactive Agentscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 22; 2024: 39; 2025: 29; 2026: 48.

138 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  5. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  6. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  7. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  8. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
  9. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  10. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  11. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  12. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
38 more
  1. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  2. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  3. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  4. 2024Video as the New Language for Real-World Decision Makingcited by 86
  5. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  6. 2024Vision Language Models are In-Context Value Learnerscited by 84
  7. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  8. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  9. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 67
  10. 2023Vision-Language Models as a Source of Rewardscited by 65
  11. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  12. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  13. 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 59
  14. 2023DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignmentcited by 58
  15. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 58
  16. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Modelscited by 54
  17. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  18. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  19. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  20. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  21. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  22. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  23. 2024Code as Reward: Empowering Reinforcement Learning with VLMscited by 38
  24. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
  25. 2024Offline Actor-Critic Reinforcement Learning Scales to Large Modelscited by 30
  26. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  27. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 29
  28. 2025Vision-Language Models for Edge Networks: A Comprehensive Surveycited by 29
  29. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 29
  30. 2025Self-Improving Embodied Foundation Modelscited by 28
  31. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  32. 2023Grounding Classical Task Planners via Vision-Language Modelscited by 23
  33. 2023Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Propertiescited by 21
  34. 2023Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake?cited by 18
  35. 2026BPP: Long-Context Robot Imitation Learning by Focusing on Key History Framescited by 17
  36. 2024Object-Centric Instruction Augmentation for Robotic Manipulationcited by 16
  37. 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planningcited by 14
  38. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 14

50 most cited of 138 All 138 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026RoboReward: General-Purpose Vision-Language Reward Models for Robotics
  2. 2025Revisiting the Learning Objectives of Vision-Language Reward Models
  3. 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models
  4. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models
  5. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics
  6. 2025TTRV: Test-Time Reinforcement Learning for Vision Language Models
  7. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Models
  8. 2026WorldGuide: Learning Success-Failure Boundaries in Latent World Models for Vision-Language-Action Policies