# Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation

arXiv: https://arxiv.org/abs/2109.13841 · HTML: https://mlsanity.com/paper/2109.13841/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 31)

- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Dynamics-Aware Unsupervised Discovery of Skills](https://mlsanity.com/paper/1907.01657/) (2019)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [Bottom-up Object Detection by Grouping Extreme and Center Points](https://mlsanity.com/paper/1901.08043/) (2019)
- [Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations](https://mlsanity.com/paper/2003.06085/) (2020)
- [AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos](https://mlsanity.com/paper/1912.04443/) (2019)
- [Efficient Parameter-free Clustering Using First Neighbor Relations](https://mlsanity.com/paper/1902.11266/) (2019)
- [Hierarchical Imitation and Reinforcement Learning](https://mlsanity.com/paper/1803.00590/) (2018)
- [IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data](https://mlsanity.com/paper/1911.05321/) (2019)
- [Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1907.13098/) (2019)
- [Neural Task Programming: Learning to Generalize Across Hierarchical Tasks](https://mlsanity.com/paper/1710.01813/) (2017)
- [TACO: Learning Task Decomposition via Temporal Alignment for Control](https://mlsanity.com/paper/1803.01840/) (2018)
- [Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation](https://mlsanity.com/paper/2103.11264/) (2021)
- [Learning Robot Skills with Temporal Variational Inference](https://mlsanity.com/paper/2006.16232/) (2020)
- [Multi-Level Discovery of Deep Options](https://mlsanity.com/paper/1703.08294/) (2017)
- [One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks](https://mlsanity.com/paper/1810.11043/) (2018)
- [Learning Multi-Arm Manipulation Through Collaborative Teleoperation](https://mlsanity.com/paper/2012.06738/) (2020)
- [SKID RAW: Skill Discovery from Raw Trajectories](https://mlsanity.com/paper/2103.14610/) (2021)
- [Modeling Long-horizon Tasks as Sequential Interaction Landscapes](https://mlsanity.com/paper/2006.04843/) (2020)
- [CompILE: Compositional Imitation Learning and Execution](https://mlsanity.com/paper/1812.01483/) (2018)
- [Action Recognition by Hierarchical Mid-level Action Elements](https://mlsanity.com/paper/1508.07654/) (2015)

## Built on by (most cited first, 30 of 83)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024)
- [SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model](https://mlsanity.com/paper/2501.15830/) (2025)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025)
- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model](https://mlsanity.com/paper/2503.10631/) (2025)
- [iVideoGPT: Interactive VideoGPTs are Scalable World Models](https://mlsanity.com/paper/2405.15223/) (2024)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning](https://mlsanity.com/paper/2410.24185/) (2024)
- [Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation](https://mlsanity.com/paper/2408.11812/) (2024)
- [Learning and Retrieval from Prior Data for Skill-based Imitation Learning](https://mlsanity.com/paper/2210.11435/) (2022)
- [XSkill: Cross Embodiment Skill Discovery](https://mlsanity.com/paper/2307.09955/) (2023)
- [IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI](https://mlsanity.com/paper/2411.00785/) (2024)
- [LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery](https://mlsanity.com/paper/2311.02058/) (2023)
- [Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons](https://mlsanity.com/paper/2603.02115/) (2026)
- [Learning Neuro-Symbolic Skills for Bilevel Planning](https://mlsanity.com/paper/2206.10680/) (2022)
- [SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment](https://mlsanity.com/paper/2410.18907/) (2024)
- [Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/2503.24361/) (2025)
- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation](https://mlsanity.com/paper/2210.12631/) (2022)
- [Eliciting Compatible Demonstrations for Multi-Human Imitation Learning](https://mlsanity.com/paper/2210.08073/) (2022)
- [MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulation](https://mlsanity.com/paper/2509.26642/) (2025)
- [Learning Options via Compression](https://mlsanity.com/paper/2212.04590/) (2022)
- [LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer](https://mlsanity.com/paper/2602.10556/) (2026)
- [CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion](https://mlsanity.com/paper/2601.09512/) (2026)
- [Learning Compositional Behaviors from Demonstration and Language](https://mlsanity.com/paper/2505.21981/) (2025)
- [XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations](https://mlsanity.com/paper/2511.02776/) (2025)

## Abstract (arXiv)

We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations and use these skills to synthesize prolonged robot behaviors. Our method starts with constructing a hierarchical task structure from each demonstration through agglomerative clustering. From the task structures of multi-task demonstrations, we identify skills based on the recurring patterns and train goal-conditioned sensorimotor policies with hierarchical imitation learning. Finally, we train a meta controller to compose these skills to solve long-horizon manipulation tasks. The entire model can be trained on a small set of human demonstrations collected within 30 minutes without further annotations, making it amendable to real-world deployment. We systematically evaluated our method in simulation environments and on a real robot. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks. Furthermore, skills discovered from multi-task demonstrations boost the average task success by 8% compared to those discovered from individual tasks.