Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
Not on the map. Why?
Abstract (arXiv)
Robot learning is splitting into two bets: policies that bake competence into frozen weights (vision-language-action, or VLA, models), and agents that write and refine their own executable skills as code. This survey organises the field around that axis of weights versus skills. Its central analytical contribution is a deep-dive that arranges code-as-policy methods by their degree of self-improvement, from zero-shot program synthesis, through closed-loop self-repair and persistent skill memory, to the sparsely populated cell in which execution feedback, skill memory, and evolutionary search combine into one open-ended loop; only a few very recent systems (for example ASPIRE, ENPIRE, and RoboClaw) occupy that cell. We map the complementary "skills" pole, from unsupervised reinforcement-learning skill discovery to large-language-model skill libraries, and show that the word "skill" is used in at least five distinct senses, of which only the code sense self-improves without gradient updates. We then connect the taxonomy to the emerging skill economy: commercial robot-skill marketplaces now distribute one-tap skills across robots but ship only static playback, which surfaces open problems of adaptation, cross-embodiment portability, provenance, safety verification, composition, and standardisation. This is a deliberately focused survey. Rather than cataloguing the field exhaustively, it examines 77 representative systems across six technique families through one taxonomy and a set of contrast tables, and it supplies operational definitions of the self-improvement mechanisms together with a statement of what each family cannot do.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 251 of its 280 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
251 of 280 references.
References · 251
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
38 more
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
- 2018World Modelscited by 1,513
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
- 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
- 2020Mastering Atari with Discrete World Modelscited by 1,007
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
- 2019Solving Rubik's Cube with a Robot Handcited by 734
- 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
- 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
- 2018Learning Dexterous In-Hand Manipulationcited by 687
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
50 most cited of 251 All 251 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026In-Context Learning for Robots: Methods and Applicationscited by 1
- 2026In-Context Learning for Robots: Methods and Applications28 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost
- 2026VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents
- 2026SEAR: Self-Evolving Agentic Robotics
- 2026Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents
- 2026SkillAdam: Stable and Efficient Skill Evolution for Agents
- 2026SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles
- 2026Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward