# Active Imitation Learning with Noisy Guidance

arXiv: https://arxiv.org/abs/2005.12801 · HTML: https://mlsanity.com/paper/2005.12801/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 26 May 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [Active Learning from Weak and Strong Labelers](https://mlsanity.com/paper/1510.02847/) (2015)
- [SEARNN: Training RNNs with Global-Local Losses](https://mlsanity.com/paper/1706.04499/) (2017)
- [Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation](https://mlsanity.com/paper/1410.5877/) (2014)
- [Replace or Retrieve Keywords In Documents at Scale](https://mlsanity.com/paper/1711.00046/) (2017)

## Built on by (most cited first, 15 of 15)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates](https://mlsanity.com/paper/2101.08133/) (2021)
- [Blending Imitation and Reinforcement Learning for Robust Policy Improvement](https://mlsanity.com/paper/2310.01737/) (2023)
- [Selective Sampling and Imitation Learning via Online Regression](https://mlsanity.com/paper/2307.04998/) (2023)
- [Towards Computationally Feasible Deep Active Learning](https://mlsanity.com/paper/2205.03598/) (2022)
- [Active Policy Improvement from Multiple Black-box Oracles](https://mlsanity.com/paper/2306.10259/) (2023)
- [Interscript: A dataset for interactive learning of scripts through error feedback](https://mlsanity.com/paper/2112.07867/) (2021)
- [On the Limitations of Simulating Active Learning](https://mlsanity.com/paper/2305.13342/) (2023)
- [Improving Neural Model Performance through Natural Language Feedback on Their Explanations](https://mlsanity.com/paper/2104.08765/) (2021)
- [Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning](https://mlsanity.com/paper/2106.09256/) (2021)
- [Interactive Machine Teaching by Labeling Rules and Instances](https://mlsanity.com/paper/2409.05199/) (2024)
- [Progressively Efficient Learning](https://mlsanity.com/paper/2310.13004/) (2023)
- [Active Third-Person Imitation Learning](https://mlsanity.com/paper/2312.16365/) (2023)
- [Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review](https://mlsanity.com/paper/2505.07911/) (2025)

## Abstract (arXiv)

Imitation learning algorithms provide state-of-the-art results on many structured prediction tasks by learning near-optimal search policies. Such algorithms assume training-time access to an expert that can provide the optimal action at any queried state; unfortunately, the number of such queries is often prohibitive, frequently rendering these approaches impractical. To combat this query complexity, we consider an active learning setting in which the learning algorithm has additional access to a much cheaper noisy heuristic that provides noisy guidance. Our algorithm, LEAQI, learns a difference classifier that predicts when the expert is likely to disagree with the heuristic, and queries the expert only when necessary. We apply LEAQI to three sequence labeling tasks, demonstrating significantly fewer queries to the expert and comparable (or better) accuracies over a passive approach.