# Interactive Imitation Learning in State-Space

arXiv: https://arxiv.org/abs/2008.00524 · HTML: https://mlsanity.com/paper/2008.00524/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 2 Aug 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation](https://mlsanity.com/paper/1703.02018/) (2017)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study](https://mlsanity.com/paper/1607.08371/) (2016)
- [Continuous Control for High-Dimensional State Spaces: An Interactive Learning Approach](https://mlsanity.com/paper/1908.05256/) (2019)

## Built on by (most cited first, 10 of 10)

- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision](https://mlsanity.com/paper/2206.14349/) (2022)
- [Interpretable Preference-based Reinforcement Learning with Tree-Structured Reward Functions](https://mlsanity.com/paper/2112.11230/) (2021)
- [Interactive incremental learning of generalizable skills with local trajectory modulation](https://mlsanity.com/paper/2409.05655/) (2024)
- [Back to Reality for Imitation Learning](https://mlsanity.com/paper/2111.12867/) (2021)
- [An Improved Data Augmentation Scheme for Model Predictive Control Policy Approximation](https://mlsanity.com/paper/2303.05607/) (2023)
- [From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback](https://mlsanity.com/paper/2502.07645/) (2025)
- [Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation](https://mlsanity.com/paper/2503.15368/) (2025)
- [Interactive Imitation Learning in Robotics based on Simulations](https://mlsanity.com/paper/2209.03900/) (2022)

## Abstract (arXiv)

Imitation Learning techniques enable programming the behavior of agents through demonstrations rather than manual engineering. However, they are limited by the quality of available demonstration data. Interactive Imitation Learning techniques can improve the efficacy of learning since they involve teachers providing feedback while the agent executes its task. In this work, we propose a novel Interactive Learning technique that uses human feedback in state-space to train and improve agent behavior (as opposed to alternative methods that use feedback in action-space). Our method titled Teaching Imitative Policies in State-space~(TIPS) enables providing guidance to the agent in terms of `changing its state' which is often more intuitive for a human demonstrator. Through continuous improvement via corrective feedback, agents trained by non-expert demonstrators using TIPS outperformed the demonstrator and conventional Imitation Learning agents.