mlsanityData to 14 Oct 2026

Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization

arXiv 2111.07999 ↗First posted 15 Nov 2021cs.LG cs.AI cs.ROcited by 46

Not on the map. Why?

Abstract (arXiv)

Skill chaining is a promising approach for synthesizing complex behaviors by sequentially combining previously learned skills. Yet, a naive composition of skills fails when a policy encounters a starting state never seen during its training. For successful skill chaining, prior approaches attempt to widen the policy's starting state distribution. However, these approaches require larger state distributions to be covered as more policies are sequenced, and thus are limited to short skill sequences. In this paper, we propose to chain multiple policies without excessively large initial state distributions by regularizing the terminal state distributions in an adversarial learning framework. We evaluate our approach on two complex long-horizon manipulation tasks of furniture assembly. Our results have shown that our method establishes the first model-free reinforcement learning algorithm to solve these tasks; whereas prior skill chaining approaches fail. The code and videos are available at https://clvrai.com/skill-chaining

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 15 Nov 2021.

Builds on

28 of 55 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  8. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  9. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  10. 2016The Option-Critic Architecturecited by 448
  11. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  12. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
16 more
  1. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  2. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  3. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  4. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  5. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  6. 2018Zero-Shot Visual Imitationcited by 117
  7. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  8. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  9. 2017Meta Learning Shared Hierarchiescited by 93
  10. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 69
  11. 2017Divide-and-Conquer Reinforcement Learningcited by 48
  12. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  13. 2018Hierarchical visuomotor control of humanoidscited by 30
  14. 2020Transferable Task Execution from Pixels through Deep Planning Domain Learningcited by 17
  15. 2021Learning Task Decomposition with Ordered Memory Policy Networkcited by 15
  16. 2015A Framework for Fine Robotic Assemblycited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 8; 2024: 11; 2025: 9; 2026: 10.

46 papers.

Sort:

By area:

  1. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 91
  2. 2022Factory: Fast Contact for Robotic Assemblycited by 54
  3. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
  4. 2022Skill-based Model-based Reinforcement Learningcited by 50
  5. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 49
  6. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulationcited by 48
  7. 2022Multi-skill Mobile Manipulation for Object Rearrangementcited by 46
  8. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  9. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 39
  10. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 27
  11. 2022Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learningcited by 22
  12. 2024ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Taskscited by 22
34 more
  1. 2024Strategy and Skill Learning for Physics-based Table Tennis Animationcited by 20
  2. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 16
  3. 2022Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learningcited by 14
  4. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
  5. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 14
  6. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 10
  7. 2025MTIL: Encoding Full History with Mamba for Temporal Imitation Learningcited by 10
  8. 2023Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robotcited by 9
  9. 2024Generalize by Touching: Tactile Ensemble Skill Transfer for Robotic Furniture Assemblycited by 7
  10. 2025BOSS: Benchmark for Observation Space Shift in Long-Horizon Taskcited by 7
  11. 2025MIMIC-D: Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policiescited by 7
  12. 2022SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Explorationcited by 6
  13. 2025Learning to Ball: Composing Policies for Long-Horizon Basketball Movescited by 6
  14. 2026Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memorycited by 5
  15. 2022Developing cooperative policies for multi-stage reinforcement learning taskscited by 4
  16. 2023Robust Subtask Learning for Compositional Generalizationcited by 4
  17. 2023Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learningcited by 4
  18. 2024Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automationcited by 4
  19. 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 4
  20. 2022Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Taskscited by 3
  21. 2024Reinforcement Learning via Auxiliary Task Distillationcited by 2
  22. 2026AtomBridge: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experimentscited by 2
  23. 2026Foresight Residual RL for Long-Horizon Robot Manipulation with Vision-Language-Action Modelscited by 2
  24. 2026EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agentscited by 2
  25. 2024Skill-Enhanced Reinforcement Learning Acceleration from Heterogeneous Demonstrationscited by 1
  26. 2026StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic Distillationcited by 1
  27. 2023Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graphcited by 0
  28. 2024Auxiliary Reward Generation with Transition Distance Representation Learningcited by 0
  29. 2025Query-Centric Diffusion Policy for Generalizable Robotic Assemblycited by 0
  30. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0
  31. 2026Skill Composition for Legged Robot Reinforcement Learningcited by 0
  32. 2026Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Videocited by 0
  33. 2026Generative Flow on Symbolic Abstractions for Embodiment Capability Discoverycited by 0
  34. 2026DexStitch: Learning and Composing Dexterous Skills from Human Videos via Contact-Aware Robotic Inpaintingcited by 0