# Explainable Hierarchical Imitation Learning for Robotic Drink Pouring

arXiv: https://arxiv.org/abs/2105.07348 · HTML: https://mlsanity.com/paper/2105.07348/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 16 May 2021.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Neural Task Programming: Learning to Generalize Across Hierarchical Tasks](https://mlsanity.com/paper/1710.01813/) (2017)
- [Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety](https://mlsanity.com/paper/1903.00640/) (2019)
- [Enhancing Explainability of Neural Networks through Architecture Constraints](https://mlsanity.com/paper/1901.03838/) (2019)
- [Visual Closed-Loop Control for Pouring Liquids](https://mlsanity.com/paper/1610.02610/) (2016)
- [Trust Considerations for Explainable Robots: A Human Factors Perspective](https://mlsanity.com/paper/2005.05940/) (2020)
- [Surgical Gesture Recognition Based on Bidirectional Multi-Layer Independently RNN with Explainable Spatial Feature Extraction](https://mlsanity.com/paper/2105.00460/) (2021)
- [Feedback Motion Planning for Liquid Transfer using Supervised Learning](https://mlsanity.com/paper/1609.03433/) (2016)

## Built on by (most cited first, 11 of 11)

- [Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2408.01366/) (2024)
- [Hierarchical Imitation Learning with Vector Quantized Models](https://mlsanity.com/paper/2301.12962/) (2023)
- [SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation](https://mlsanity.com/paper/2504.15561/) (2025)
- [One-Shot Domain-Adaptive Imitation Learning via Progressive Learning](https://mlsanity.com/paper/2204.11251/) (2022)
- [Cloud-Based Hierarchical Imitation Learning for Scalable Transfer of Construction Skills from Human Workers to Assisting Robots](https://mlsanity.com/paper/2309.11619/) (2023)
- [Explaining Imitation Learning through Frames](https://mlsanity.com/paper/2301.01088/) (2023)
- [Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies](https://mlsanity.com/paper/2309.16025/) (2023)
- [Offline Imitation of Badminton Player Behavior via Experiential Contexts and Brownian Motion](https://mlsanity.com/paper/2403.12406/) (2024)
- [Enabling robots to follow abstract instructions and complete complex dynamic tasks](https://mlsanity.com/paper/2406.11231/) (2024)
- [CasIL: Cognizing and Imitating Skills via a Dual Cognition-Action Architecture](https://mlsanity.com/paper/2309.16299/) (2023)
- [Interpretable Robotic Manipulation from Language](https://mlsanity.com/paper/2405.17047/) (2024)

## Abstract (arXiv)

To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with EHIL, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability and explainability.