mlsanityData to 7 Oct 2026

Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation

arXiv 2109.01115 ↗First posted 2 Sep 2021cs.RO cs.AI cs.LGcited by 117

Not on the map. Why?

Abstract (arXiv)

We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction. In order to accomplish this, humans need easy and effective ways of specifying tasks to the robot. Goal images are one popular form of task specification, as they are already grounded in the robot's observation space. However, goal images also have a number of drawbacks: they are inconvenient for humans to provide, they can over-specify the desired behavior leading to a sparse reward signal, or under-specify task information in the case of non-goal reaching tasks. Natural language provides a convenient and flexible alternative for task specification, but comes with the challenge of grounding language in the robot's observation space. To scalably learn this grounding we propose to leverage offline robot datasets (including highly sub-optimal, autonomously collected data) with crowd-sourced natural language labels. With this data, we learn a simple classifier which predicts if a change in state completes a language instruction. This provides a language-conditioned reward function that can then be used for offline multi-task RL. In our experiments, we find that on language-conditioned manipulation tasks our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25%, and is able to perform visuomotor tasks from natural language, such as "open the right drawer" and "move the stapler", on a Franka Emika Panda robot.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Builds on 62 of its 80 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

62 of 80 references.

References · 62

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  9. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  10. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  11. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
38 more
  1. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  2. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  3. 2017Stochastic Variational Video Predictioncited by 234
  4. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  5. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  6. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  7. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  8. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  9. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  10. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  11. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  12. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  13. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  14. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  15. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  16. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  17. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  18. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  19. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  20. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  21. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 80
  22. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  23. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  24. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  25. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  26. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  27. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  28. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  29. 2016Learning Language Games through Interactioncited by 43
  30. 2019Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flightcited by 43
  31. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  32. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  33. 2018Guiding Policies with Language via Meta-Learningcited by 34
  34. 2020PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewardscited by 30
  35. 2020Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planningcited by 26
  36. 2017Beating Atari with Natural Language Guided Reinforcement Learningcited by 24
  37. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  38. 2020MELD: Meta-Reinforcement Learning from Images via Latent State Modelscited by 22

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 21; 2023: 40; 2024: 33; 2025: 18; 2026: 4.

117 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  6. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  8. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  9. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  10. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  11. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  12. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
38 more
  1. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  4. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  5. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  6. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  7. 2023Language-Driven Representation Learning for Roboticscited by 163
  8. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  9. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
  10. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  11. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  12. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  13. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
  14. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  15. 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
  16. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidancecited by 100
  17. 2022Benchmarking Spatial Relationships in Text-to-Image Generationcited by 96
  18. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  19. 2023Preference Transformer: Modeling Human Preferences using Transformers for RLcited by 77
  20. 2024HiRT: Enhancing Robotic Control with Hierarchical Robot Transformerscited by 72
  21. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  22. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  23. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 66
  24. 2023Learning to Model the World with Languagecited by 66
  25. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  26. 2023DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignmentcited by 58
  27. 2022StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objectscited by 49
  28. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  29. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  30. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  31. 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
  32. 2023Learning adaptive planning representations with natural language guidancecited by 36
  33. 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
  34. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agentscited by 32
  35. 2023Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Controlcited by 32
  36. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 32
  37. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  38. 2022LISA: Learning Interpretable Skill Abstractions from Languagecited by 30

50 most cited of 117 All 117 (JSON)