# Interactive Policy Learning through Confidence-Based Autonomy

arXiv: https://arxiv.org/abs/1401.3439 · HTML: https://mlsanity.com/paper/1401.3439/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2014.

## Built on by (most cited first, 29 of 29)

- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision](https://mlsanity.com/paper/2206.14349/) (2022)
- [IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning](https://mlsanity.com/paper/2405.01472/) (2024)
- [Uncertainty-Aware Data Aggregation for Deep Imitation Learning](https://mlsanity.com/paper/1905.02780/) (2019)
- [Interactive Imitation Learning in State-Space](https://mlsanity.com/paper/2008.00524/) (2020)
- [Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism](https://mlsanity.com/paper/2506.09176/) (2025)
- [SAFARI: Safe and Active Robot Imitation Learning with Imagination](https://mlsanity.com/paper/2011.09586/) (2020)
- [Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy](https://mlsanity.com/paper/2102.09854/) (2021)
- [A Framework for Learning from Demonstration with Minimal Human Effort](https://mlsanity.com/paper/2306.09211/) (2023)
- [An Improved Data Augmentation Scheme for Model Predictive Control Policy Approximation](https://mlsanity.com/paper/2303.05607/) (2023)
- [Predictive Preference Learning from Human Interventions](https://mlsanity.com/paper/2510.01545/) (2025)
- [Learning User's confidence for active learning](https://mlsanity.com/paper/2104.07791/) (2021)
- [Human-In-The-Loop Document Layout Analysis](https://mlsanity.com/paper/2108.02095/) (2021)
- [Improving Robot-Centric Learning from Demonstration via Personalized Embeddings](https://mlsanity.com/paper/2110.03134/) (2021)
- ["Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2406.03069/) (2024)
- [Continual Robot Skill and Task Learning via Dialogue](https://mlsanity.com/paper/2409.03166/) (2024)
- [Uncertainty Aware Learning from Demonstrations in Multiple Contexts using Bayesian Neural Networks](https://mlsanity.com/paper/1903.05697/) (2019)
- [Generalizing to New Tasks via One-Shot Compositional Subgoals](https://mlsanity.com/paper/2205.07716/) (2022)
- [An Integrated Imitation and Reinforcement Learning Methodology for Robust Agile Aircraft Control with Limited Pilot Demonstration Data](https://mlsanity.com/paper/2401.08663/) (2023)
- [Learning Gaussian Policies from Corrective Human Feedback](https://mlsanity.com/paper/1903.05216/) (2019)
- [Investigating the Effects of Robot Engagement Communication on Learning from Demonstration](https://mlsanity.com/paper/2005.01020/) (2020)
- [Human-Machine Teaming for UAVs: An Experimentation Platform](https://mlsanity.com/paper/2312.11718/) (2023)
- [Learning from Demonstration Framework for Multi-Robot Systems Using Interaction Keypoints and Soft Actor-Critic Methods](https://mlsanity.com/paper/2404.02324/) (2024)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [YRC-Bench: A Benchmark for Learning to Coordinate with Experts](https://mlsanity.com/paper/2502.09583/) (2025)
- [Addressing Ambiguity in Imitation Learning through Product of Experts based Negative Feedback](https://mlsanity.com/paper/2603.26467/) (2026)
- [Deliberate Practice: Learning Robot Skills under a Budget](https://mlsanity.com/paper/2608.13415/) (2026)

## Abstract (arXiv)

We present Confidence-Based Autonomy (CBA), an interactive algorithm for policy learning from demonstration. The CBA algorithm consists of two components which take advantage of the complimentary abilities of humans and computer agents. The first component, Confident Execution, enables the agent to identify states in which demonstration is required, to request a demonstration from the human teacher and to learn a policy based on the acquired data. The algorithm selects demonstrations based on a measure of action selection confidence, and our results show that using Confident Execution the agent requires fewer demonstrations to learn the policy than when demonstrations are selected by a human teacher. The second algorithmic component, Corrective Demonstration, enables the teacher to correct any mistakes made by the agent through additional demonstrations in order to improve the policy and future task performance. CBA and its individual components are compared and evaluated in a complex simulated driving domain. The complete CBA algorithm results in the best overall learning performance, successfully reproducing the behavior of the teacher while balancing the tradeoff between number of demonstrations and number of incorrect actions during learning.