# Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation

arXiv: https://arxiv.org/abs/2404.04219 · HTML: https://mlsanity.com/paper/2404.04219/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 10 of 10)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges](https://mlsanity.com/paper/1907.00182/) (2019)
- [DisCoRL: Continual Reinforcement Learning via Policy Distillation](https://mlsanity.com/paper/1907.05855/) (2019)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [Ex-Model: Continual Learning from a Stream of Trained Models](https://mlsanity.com/paper/2112.06511/) (2021)
- [RBO Hand 3 -- A Platform for Soft Dexterous Manipulation](https://mlsanity.com/paper/2201.10883/) (2022)
- [Plant-inspired behavior-based controller to enable reaching in redundant continuum robot arms](https://mlsanity.com/paper/2304.07188/) (2023)

## Built on by (most cited first, 4 of 4)

- [Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey](https://mlsanity.com/paper/2506.00098/) (2025)
- [Continual Policy Consolidation for Lifelong Robot Learning](https://mlsanity.com/paper/2601.22475/) (2026)
- [FT-NCFM: An Influence-Aware Data Distillation Framework for Efficient VLA Models](https://mlsanity.com/paper/2511.16233/) (2025)
- [GEM-Style Constraints for PEFT with Dual Gradient Projection in LoRA](https://mlsanity.com/paper/2601.02500/) (2026)

## Abstract (arXiv)

Dexterous manipulation, often facilitated by multi-fingered robotic hands, holds solid impact for real-world applications. Soft robotic hands, due to their compliant nature, offer flexibility and adaptability during object grasping and manipulation. Yet, benefits come with challenges, particularly in the control development for finger coordination. Reinforcement Learning (RL) can be employed to train object-specific in-hand manipulation policies, but limiting adaptability and generalizability. We introduce a Continual Policy Distillation (CPD) framework to acquire a versatile controller for in-hand manipulation, to rotate different objects in shape and size within a four-fingered soft gripper. The framework leverages Policy Distillation (PD) to transfer knowledge from expert policies to a continually evolving student policy network. Exemplar-based rehearsal methods are then integrated to mitigate catastrophic forgetting and enhance generalization. The performance of the CPD framework over various replay strategies demonstrates its effectiveness in consolidating knowledge from multiple experts and achieving versatile and adaptive behaviours for in-hand manipulation tasks.