SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation
On the map: LLM agents and tool use → Agent skill learning and evolution
Abstract (arXiv)
Large-scale demonstrations have driven unprecedented progress in robot learning, yet collecting robot data through teleoperation is expensive and difficult to scale to diverse environments and long-horizon tasks. Simulation offers a scalable alternative, but existing data-generation pipelines often rely on open-loop controllers, scripted skill sequences, or task-specific programs. We introduce SkillWeaver, an agentic framework that autonomously generates robot experience by exploring over Neural Interaction Skills (NIS): reusable, parameterized, closed-loop policies that expose learned physical interaction capabilities to a reasoning agent. Given a task and a simulated environment, a VLM agent reasons about what to do next, invokes and parameterizes NIS to interact with the environment, observes their outcomes, and generates verification, reflection, and memory to guide subsequent exploration. We instantiate NIS as reinforcement-learned policies for closed-loop, contact-rich manipulation and organize exploration as verifier-guided tree search, enabling the agent to discover successful long-horizon behaviors without relying on predetermined execution pipelines. SkillWeaver scales autonomously to 39.1K demonstrations across 14.1K scenes, which we distill into visuomotor policies. Across simulation benchmarks and real-world manipulation, training on SkillWeaver-generated experience substantially improves generalization to novel objects, spatial configurations, tasks, and environments, and enables zero- and few-shot sim-to-sim and sim-to-real transfer. Our results suggest agentic exploration over neural interaction skills as a scalable alternative for robot data generation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 30 of its 48 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
30 of 48 references.
References · 30
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
- 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
- 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
18 more
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
- 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 179
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 125
- 2024CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Modelscited by 99
- 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 88
- 2026ASPIRE: Agentic /Skills Discovery for Roboticscited by 53
- 2023cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generationcited by 52
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 51
- 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 50
- 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
- 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Braincited by 25
- 2025Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learningcited by 24
- 2026Playful Agentic Robot Learningcited by 22
- 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenescited by 20
- 2025One View, Many Worlds: Single-Image to 3D Object Meets Generative Domain Randomization for One-Shot 6D Pose Estimationcited by 14
- 2026Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robotcited by 14
- 2026MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulationcited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents
- 2026Teach and Grow: An Agent-Centered Architecture for General Robot Learning
- 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
- 2026ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation
- 2026Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation
- 2024Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation
- 2026Nautilus: From One Prompt to Plug-and-Play Robot Learning
- 2026EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution