mlsanityData to 14 Oct 2026

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

arXiv 1710.01813 ↗First posted 4 Oct 2017cs.AI cs.LG cs.ROcited by 76

Not on the map. Why?

Cited by 21 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well to- wards unseen tasks with increasing lengths, variable topologies, and changing objectives.

Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). First posted on arXiv 4 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 5; 2020: 0; 2021: 16; 2022: 20; 2023: 10; 2024: 13; 2025: 4; 2026: 6.

76 papers.

Sort:

By area:

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  3. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  4. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  5. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  6. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 83
  7. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 82
  8. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  9. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
  10. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  11. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
  12. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 70
38 more
  1. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 67
  2. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  3. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  4. 2022TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphscited by 59
  5. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 49
  6. 2024Efficient Data Collection for Robotic Manipulation via Compositional Generalizationcited by 37
  7. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 35
  8. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 34
  9. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Taskscited by 33
  10. 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulationcited by 32
  11. 2019Few-Shot Bayesian Imitation Learning with Logical Program Policiescited by 30
  12. 2022LISA: Learning Interpretable Skill Abstractions from Languagecited by 30
  13. 2022Chain of Thought Imitation with Procedure Cloningcited by 30
  14. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  15. 2023Universal Visual Decomposer: Long-Horizon Manipulation Made Easycited by 25
  16. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replaycited by 21
  17. 2022Generating Executable Action Plans with Environmentally-Aware Language Modelscited by 21
  18. 2024A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approachescited by 20
  19. 2022STAP: Sequencing Task-Agnostic Policiescited by 18
  20. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 18
  21. 2023Chain-of-Thought Predictive Controlcited by 18
  22. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  23. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 13
  24. 2023Multimodal Subtask Graph Generation from Instructional Videoscited by 13
  25. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  26. 2021Explainable Hierarchical Imitation Learning for Robotic Drink Pouringcited by 11
  27. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 11
  28. 2019To Follow or not to Follow: Selective Imitation Learning from Observationscited by 10
  29. 2022Transporters with Visual Foresight for Solving Unseen Rearrangement Taskscited by 9
  30. 2019Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learningcited by 8
  31. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrationscited by 7
  32. 2023Hierarchical Neural Program Synthesiscited by 7
  33. 2024One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skillcited by 7
  34. 2024DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignmentcited by 7
  35. 2025Structured Preference Optimization for Vision-Language Long-Horizon Task Planningcited by 7
  36. 2026Evolving Programmatic Skill Networkscited by 7
  37. 2019Motion Reasoning for Goal-Based Imitation Learningcited by 6
  38. 2024Task and Motion Planning for Execution in the Realcited by 6

50 most cited of 76 All 76 (JSON)