# Cloud-Based Hierarchical Imitation Learning for Scalable Transfer of Construction Skills from Human Workers to Assisting Robots

arXiv: https://arxiv.org/abs/2309.11619 · HTML: https://mlsanity.com/paper/2309.11619/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 92 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 15 of 15)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Parrot: Data-Driven Behavioral Priors for Reinforcement Learning](https://mlsanity.com/paper/2011.10024/) (2020)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Multi-Stage Cable Routing through Hierarchical Imitation Learning](https://mlsanity.com/paper/2307.08927/) (2023)
- [Deep Imitation Learning for Bimanual Robotic Manipulation](https://mlsanity.com/paper/2010.05134/) (2020)
- [DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools](https://mlsanity.com/paper/2203.17275/) (2022)
- [Transferable Task Execution from Pixels through Deep Planning Domain Learning](https://mlsanity.com/paper/2003.03726/) (2020)
- [Multi-task Hierarchical Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/2305.12633/) (2023)
- [An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications](https://mlsanity.com/paper/2203.08619/) (2022)
- [Explainable Hierarchical Imitation Learning for Robotic Drink Pouring](https://mlsanity.com/paper/2105.07348/) (2021)
- [Robotic Imitation of Human Assembly Skills Using Hybrid Trajectory and Force Learning](https://mlsanity.com/paper/2103.05912/) (2021)
- [One-Shot Domain-Adaptive Imitation Learning via Progressive Learning](https://mlsanity.com/paper/2204.11251/) (2022)
- [Direct off-line robot programming via a common CAD package](https://mlsanity.com/paper/1309.2078/) (2013)
- [Enabling Building Information Model-Driven Human-Robot Collaborative Construction Workflows with Closed-Loop Digital Twins](https://mlsanity.com/paper/2306.09639/) (2023)

## Built on by (most cited first, 2 of 2)

- [Building Information Models to Robot-Ready Site Digital Twins (BIM2RDT): An Agentic AI Safety-First Framework](https://mlsanity.com/paper/2509.20705/) (2025)
- [Sample-Efficient Robot Skill Learning for Construction Tasks: Benchmarking Hierarchical Reinforcement Learning and Vision-Language-Action VLA Model](https://mlsanity.com/paper/2512.14031/) (2025)

## Abstract (arXiv)

Assigning repetitive and physically-demanding construction tasks to robots can alleviate human workers's exposure to occupational injuries. Transferring necessary dexterous and adaptive artisanal construction craft skills from workers to robots is crucial for the successful delegation of construction tasks and achieving high-quality robot-constructed work. Predefined motion planning scripts tend to generate rigid and collision-prone robotic behaviors in unstructured construction site environments. In contrast, Imitation Learning (IL) offers a more robust and flexible skill transfer scheme. However, the majority of IL algorithms rely on human workers to repeatedly demonstrate task performance at full scale, which can be counterproductive and infeasible in the case of construction work. To address this concern, this paper proposes an immersive, cloud robotics-based virtual demonstration framework that serves two primary purposes. First, it digitalizes the demonstration process, eliminating the need for repetitive physical manipulation of heavy construction objects. Second, it employs a federated collection of reusable demonstrations that are transferable for similar tasks in the future and can thus reduce the requirement for repetitive illustration of tasks by human agents. Additionally, to enhance the trustworthiness, explainability, and ethical soundness of the robot training, this framework utilizes a Hierarchical Imitation Learning (HIL) model to decompose human manipulation skills into sequential and reactive sub-skills. These two layers of skills are represented by deep generative models, enabling adaptive control of robot actions. By delegating the physical strains of construction work to human-trained robots, this framework promotes the inclusion of workers with diverse physical capabilities and educational backgrounds within the construction industry.