mlsanityData to 7 Oct 2026

Autonomous Improvement of Instruction Following Skills via Foundation Models

arXiv 2407.20635 ↗First posted 30 Jul 2024cs.RO cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

Intelligent instruction-following robots capable of improving from autonomously collected experience have the potential to transform robot learning: instead of collecting costly teleoperated demonstration data, large-scale deployment of fleets of robots can quickly collect larger quantities of autonomous data that can collectively improve their performance. However, autonomous improvement requires solving two key problems: (i) fully automating a scalable data collection procedure that can collect diverse and semantically meaningful robot data and (ii) learning from non-optimal, autonomous data with no human annotations. To this end, we propose a novel approach that addresses these challenges, allowing instruction-following policies to improve from autonomously collected data without human supervision. Our framework leverages vision-language models to collect and evaluate semantically meaningful experiences in new environments, and then utilizes a decomposition of instruction following tasks into (semantic) language-conditioned image generation and (non-semantic) goal reaching, which makes it significantly more practical to improve from this autonomously collected data without any human annotations. We carry out extensive experiments in the real world to demonstrate the effectiveness of our approach, and find that in a suite of unseen environments, the robot policy can be improved 2x with autonomously collected data. We open-source the code for our semantic autonomous improvement pipeline, as well as our autonomous dataset of 30.5K trajectories collected across five tabletop environments.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 79 of its 95 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

79 of 95 references.

References · 79

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  12. 2015VQA: Visual Question Answeringcited by 2,489
38 more
  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  5. 2019Towards VQA Models That Can Readcited by 1,696
  6. 2018Group Normalizationcited by 1,514
  7. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  9. 2023PaLM 2 Technical Reportcited by 1,154
  10. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  11. 2017Hindsight Experience Replaycited by 1,096
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  13. 2022A Generalist Agentcited by 802
  14. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  15. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  16. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  17. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  18. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  19. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  20. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  21. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  22. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  23. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  24. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  25. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  26. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  27. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  28. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  29. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  30. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  31. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  32. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  33. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  34. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
  35. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  36. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  37. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 151
  38. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 14.

32 papers.

Sort:

By area:

  1. 2025Training Agents Inside of Scalable World Modelscited by 122
  2. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RLcited by 72
  3. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  4. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  5. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
  6. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  7. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  8. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  9. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  10. 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Modelscited by 21
  11. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 19
  12. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
20 more
  1. 2026PlayWorld: Learning Robot World Models from Autonomous Playcited by 16
  2. 2025RobotArena ∞: Scalable Robot Benchmarking via Real-to-Sim Translationcited by 15
  3. 2024GHIL-Glue: Hierarchical Control with Filtered Subgoal Imagescited by 14
  4. 2025What Matters for Batch Online Reinforcement Learning in Robotics?cited by 14
  5. 2025Multimodal Spatial Language Maps for Robot Navigation and Manipulationcited by 6
  6. 2025Improving Robotic Manipulation Robustness via NICE Scene Surgerycited by 3
  7. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2
  8. 2025Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Searchcited by 2
  9. 2026Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warpingcited by 2
  10. 2026Robot Self-Improvement via Human-Video Dynamics Modelscited by 2
  11. 2026RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Resetcited by 1
  12. 2025CroSTAta: Cross-State Transition Attention Transformer for Robotic Manipulationcited by 0
  13. 2026Learning More from Less: Reinforcement Learning from Hindsightcited by 0
  14. 2026HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentationcited by 0
  15. 2026FailBench: How Reliable are VLMs at Judging Robot Task Success?cited by 0
  16. 2026Subspace Inference Enables Efficient Active Reward Learning from Preferencescited by 0
  17. 2026Dissecting Advantage-Guided Post-Training for Vision-Language-Action Policiescited by 0
  18. 2026SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptationcited by 0
  19. 2026Skill-Space Shooting for Autonomous Robot Policy Improvementcited by 0
  20. 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control
  2. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement
  3. 2024Learning Novel Skills from Language-Generated Demonstrations
  4. 2023Scaling Robot Learning with Semantically Imagined Experience
  5. 2026InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulation
  6. 2024The intrinsic motivation of reinforcement and imitation learning for sequential tasks
  7. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy
  8. 2024RoboDreamer: Learning Compositional World Models for Robot Imagination