# Imitating Unknown Policies via Exploration

arXiv: https://arxiv.org/abs/2008.05660 · HTML: https://mlsanity.com/paper/2008.05660/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 13 Aug 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 7 of 7)

- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Learning from Visual Observation via Offline Pretrained State-to-Go Transformer](https://mlsanity.com/paper/2306.12860/) (2023)
- [Explorative Imitation Learning: A Path Signature Approach for Continuous Environments](https://mlsanity.com/paper/2407.04856/) (2024)
- [NOLO: Navigate Only Look Once](https://mlsanity.com/paper/2408.01384/) (2024)
- [Make Bipedal Robots Learn How to Imitate](https://mlsanity.com/paper/2105.07193/) (2021)
- [Self-Supervised Adversarial Imitation Learning](https://mlsanity.com/paper/2304.10914/) (2023)
- [Quantifying Generalisation in Imitation Learning](https://mlsanity.com/paper/2509.24784/) (2025)

## Abstract (arXiv)

Behavioral cloning is an imitation learning technique that teaches an agent how to behave through expert demonstrations. Recent approaches use self-supervision of fully-observable unlabeled snapshots of the states to decode state-pairs into actions. However, the iterative learning scheme from these techniques are prone to getting stuck into bad local minima. We address these limitations incorporating a two-phase model into the original framework, which learns from unlabeled observations via exploration, substantially improving traditional behavioral cloning by exploiting (i) a sampling mechanism to prevent bad local minima, (ii) a sampling mechanism to improve exploration, and (iii) self-attention modules to capture global features. The resulting technique outperforms the previous state-of-the-art in four different environments by a large margin.