mlsanityData to 7 Oct 2026

SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation

arXiv 2410.18065 ↗First posted 23 Oct 2024cs.RO cs.AI cs.CVcited by 8

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Robot learning has proven to be a general and effective technique for programming manipulators. Imitation learning is able to teach robots solely from human demonstrations but is bottlenecked by the capabilities of the demonstrations. Reinforcement learning uses exploration to discover better behaviors; however, the space of possible improvements can be too large to start from scratch. And for both techniques, the learning difficulty increases proportional to the length of the manipulation task. Accounting for this, we propose SPIRE, a system that first uses Task and Motion Planning (TAMP) to decompose tasks into smaller learning subproblems and second combines imitation and reinforcement learning to maximize their strengths. We develop novel strategies to train learning agents when deployed in the context of a planning system. We evaluate SPIRE on a suite of long-horizon and contact-rich robot manipulation problems. We find that SPIRE outperforms prior approaches that integrate imitation learning, reinforcement learning, and planning by 35% to 50% in average task performance, is 6 times more data efficient in the number of human demonstrations needed to train proficient agents, and learns to complete tasks nearly twice as efficiently. View https://sites.google.com/view/spire-corl-2024 for more details.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 50 of its 66 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

50 of 66 references.

Closest ancestors · 7

Same direction, newest first

  1. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations6 Mar
  2. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  3. 2023Human-in-the-Loop Task and Motion Planning for Imitation Learning24 Oct
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  5. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations2 Mar
  6. 2022Behavior Transformers: Cloning k modes with one stone22 Jun
  7. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb

Other references · 43

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  9. 2021Implicit Behavioral Cloningcited by 442
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  11. 2016The Option-Critic Architecturecited by 433
  12. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
31 more
  1. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  2. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  3. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  4. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  5. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  6. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  7. 2020Integrated Task and Motion Planningcited by 281
  8. 2020First return, then explorecited by 256
  9. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  10. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  11. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  12. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  13. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  14. 2024Behavior Generation with Latent Actionscited by 159
  15. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  16. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
  17. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  18. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  19. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  20. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  21. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  22. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  23. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 51
  24. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  25. 2023IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Realitycited by 44
  26. 2024RePLan: Robotic Replanning with Perception and Language Modelscited by 39
  27. 2022Causal Imitation Learning under Temporally Correlated Noisecited by 28
  28. 2017Deep Q-learning from Demonstrationscited by 27
  29. 2020Meta Reinforcement Learning with Autonomous Inference of Subtask Dependenciescited by 25
  30. 2018Learning Representations in Model-Free Hierarchical Reinforcement Learningcited by 13
  31. 2022Possibility Before Utility: Learning And Using Hierarchical Affordancescited by 4

Built on by

Cited by year

Citing papers per year: 2025: 8.

8 papers.

  1. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulationcited by 48
  2. 2025FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokenscited by 9
  3. 2025Hybrid Diffusion for Simultaneous Symbolic and Continuous Planningcited by 6
  4. 2025Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingcited by 4
  5. 2025ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learningcited by 3
  6. 2025FoldNet: Learning Generalizable Closed-Loop Policy for Garment Folding via Keypoint-Driven Asset and Demonstration Synthesiscited by 1
  7. 2025Risk-Sensitive Agent Compositionscited by 1
  8. 2025Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skillscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  2. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
  3. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
  4. 2026Beyond the Majority: Long-tail Imitation Learning for Robotic Manipulation
  5. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
  6. 2023Offline Skill Generalization via Task and Motion Planning
  7. 2020Modeling Long-horizon Tasks as Sequential Interaction Landscapes
  8. 2024PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning