mlsanityData to 14 Oct 2026

Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance

arXiv 2310.10021 ↗First posted 16 Oct 2023cs.RO cs.AI cs.LGcited by 86

On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents

Abstract (arXiv)

We propose BOSS, an approach that automatically learns to solve new long-horizon, complex, and meaningful tasks by growing a learned skill library with minimal supervision. Prior work in reinforcement learning require expert supervision, in the form of demonstrations or rich reward functions, to learn long-horizon tasks. Instead, our approach BOSS (BOotStrapping your own Skills) learns to accomplish new tasks by performing "skill bootstrapping," where an agent with a set of primitive skills interacts with the environment to practice new skills without receiving reward feedback for tasks outside of the initial skill set. This bootstrapping phase is guided by large language models (LLMs) that inform the agent of meaningful skills to chain together. Through this process, BOSS builds a wide range of complex and useful behaviors from a basic set of primitive skills. We demonstrate through experiments in realistic household environments that agents trained with our LLM-guided bootstrapping procedure outperform those trained with naive bootstrapping as well as prior unsupervised skill acquisition methods on zero-shot execution of unseen, long-horizon tasks in new environments. Website at clvrai.com/boss.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 58 of its 69 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

58 of 69 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2016Layer Normalizationcited by 5,669
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  11. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
38 more
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  4. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  5. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  9. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  10. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  11. 2016The Option-Critic Architecturecited by 448
  12. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  13. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  14. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  15. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  16. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  17. 2016Variational Intrinsic Controlcited by 267
  18. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  19. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  20. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  21. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  22. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  23. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  24. 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
  25. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  26. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 133
  27. 2021Discovering and Achieving Goals via World Modelscited by 131
  28. 2018Variational Option Discovery Algorithmscited by 122
  29. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  30. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  31. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  32. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 86
  33. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  34. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 71
  35. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  36. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 68
  37. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  38. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 58

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 41; 2025: 22; 2026: 19.

86 papers.

Sort:

By area:

  1. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  2. 2024Large Multimodal Agents: A Surveycited by 105
  3. 2024Vision Language Models are In-Context Value Learnerscited by 90
  4. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  5. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 83
  6. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 57
  7. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  8. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  9. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulationcited by 48
  10. 2026Code as Agent Harnesscited by 47
  11. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  12. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
38 more
  1. 2025CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Modelscited by 29
  2. 2025A Survey of Personalization: From RAG to Agentcited by 27
  3. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  4. 2024Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learningcited by 23
  5. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 22
  6. 2024AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulationcited by 20
  7. 2026Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Controlcited by 20
  8. 2025A Survey of Continual Reinforcement Learningcited by 18
  9. 2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 16
  10. 2024AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulationcited by 16
  11. 2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
  12. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  13. 2025Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learningcited by 14
  14. 2024LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Surveycited by 12
  15. 2024RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Modelscited by 12
  16. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 12
  17. 2025PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policiescited by 12
  18. 2024Language-guided Skill Learning with Temporal Variational Inferencecited by 11
  19. 2025BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generationcited by 11
  20. 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
  21. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learningcited by 10
  22. 2024Enhancing Agent Learning through World Dynamics Modelingcited by 9
  23. 2024Defining and Evaluating Physical Safety for Large Language Modelscited by 9
  24. 2024MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memorycited by 9
  25. 2025SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Modelscited by 9
  26. 2024Retrieval-Augmented Hierarchical in-Context Reinforcement Learning and Hindsight Modular Reflections for Task Planning with LLMscited by 8
  27. 2023LHManip: A Dataset for Long-Horizon Language-Grounded Manipulation Tasks in Cluttered Tabletop Environmentscited by 7
  28. 2024HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulationcited by 7
  29. 2024Tulip Agent -- Enabling LLM-Based Agents to Solve Tasks Using Large Tool Librariescited by 7
  30. 2024Generative AI Agents in Autonomous Machines: A Safety Perspectivecited by 7
  31. 2025BOSS: Benchmark for Observation Space Shift in Long-Horizon Taskcited by 7
  32. 2024Multi-granularity Knowledge Transfer for Continual Reinforcement Learningcited by 6
  33. 2024World Models with Hints of Large Language Models for Goal Achievingcited by 6
  34. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 6
  35. 2024Agentic Skill Discoverycited by 5
  36. 2025Schema-Guided Scene-Graph Reasoning based on Multi-Agent Large Language Model Systemcited by 5
  37. 2023Location-Aware Visual Question Generation with Lightweight Modelscited by 4
  38. 2024Large Language Model Agents for Improving Engagement with Behavior Change Interventions: Application to Digital Mindfulnesscited by 4

50 most cited of 86 All 86 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026SKILLER: Language-Level Reinforcement Learning for Reusable Skill Extraction in Small Language Models
  2. 2026Search2Skill: Skill Distillation Beyond Knowledge Boundaries Via Rubric-Based Reinforcement Learning
  3. 2026SkillForge: Evolving Verifiable Skills for Reinforcement Learning Agents
  4. 2026SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
  5. 2026Context Bootstrapped Reinforcement Learning
  6. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
  7. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learning
  8. 2025Bootstrapping Task Spaces for Self-Improvement