mlsanityData to 7 Oct 2026

CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observations

arXiv 2311.03485 ↗First posted 6 Nov 2023cs.RO cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel method for learning reward functions for robotic motions by harnessing the power of a CLIP-based model. Traditional reward function design often hinges on manual feature engineering, which can struggle to generalize across an array of tasks. Our approach circumvents this challenge by capitalizing on CLIP's capability to process both state features and image inputs effectively. Given a pair of consecutive observations, our model excels in identifying the motion executed between them. We showcase results spanning various robotic activities, such as directing a gripper to a designated target and adjusting the position of a cube. Through experimental evaluations, we underline the proficiency of our method in precisely deducing motion and its promise to enhance reinforcement learning training in the realm of robotics.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 47 of its 66 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

47 of 66 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2016Layer Normalizationcited by 5,579
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2015Continuous control with deep reinforcement learningcited by 3,279
  11. 2023Mistral 7Bcited by 3,065
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
35 more
  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  10. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  11. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  12. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  13. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  14. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  15. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  16. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  17. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  18. 2023Vision-Language Models as Success Detectorscited by 138
  19. 2021Discovering and Achieving Goals via World Modelscited by 131
  20. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  21. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  22. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  23. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
  24. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  25. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  26. 2019Learning Reward Functions by Integrating Human Demonstrations and Preferencescited by 64
  27. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  28. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  29. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  30. 2018A Hierarchical Framework for Relation Extraction with Reinforcement Learningcited by 49
  31. 2018Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependenciescited by 43
  32. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  33. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  34. 2018Few-Shot Goal Inference for Visuomotor Learning and Planningcited by 30
  35. 2021MHER: Model-based Hindsight Experience Replaycited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  2. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  3. 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learningcited by 2
  4. 2025Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Modelscited by 2
  5. 2026MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Modelscited by 2
  6. 2025Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learningcited by 1
  7. 2024STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in Minecraftcited by 0
  8. 2026Dreaming Across Towns: Semantic Rollout and Town-Adversarial Regularization for Zero-Shot Held-Out-Town Fixed-Route Driving in CARLAcited by 0