# Behavioral Cloning via Search in Embedded Demonstration Dataset

arXiv: https://arxiv.org/abs/2306.09082 · HTML: https://mlsanity.com/paper/2306.09082/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned](https://mlsanity.com/paper/2202.10583/) (2022)
- [The MineRL BASALT Competition on Learning from Human Feedback](https://mlsanity.com/paper/2107.01969/) (2021)
- [Contrastive Language, Action, and State Pre-training for Robot Learning](https://mlsanity.com/paper/2304.10782/) (2023)
- [Benchmarking End-to-End Behavioural Cloning on Video Games](https://mlsanity.com/paper/2004.00981/) (2020)
- [Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition](https://mlsanity.com/paper/2303.13512/) (2023)
- [Robust Behavioral Cloning for Autonomous Vehicles using End-to-End Imitation Learning](https://mlsanity.com/paper/2010.04767/) (2020)
- [Playing Minecraft with Behavioural Cloning](https://mlsanity.com/paper/2005.03374/) (2020)
- [Planning with RL and episodic-memory behavioral priors](https://mlsanity.com/paper/2207.01845/) (2022)
- [Solving Learn-to-Race Autonomous Racing Challenge by Planning in Latent Space](https://mlsanity.com/paper/2207.01275/) (2022)

## Built on by (most cited first, 3 of 3)

- [UniTeam: Open Vocabulary Mobile Manipulation Challenge](https://mlsanity.com/paper/2312.08611/) (2023)
- [Benchmarks for Physical Reasoning AI](https://mlsanity.com/paper/2312.10728/) (2023)
- [Language-Conditioned Semantic Search-Based Policy for Robotic Manipulation Tasks](https://mlsanity.com/paper/2312.05925/) (2023)

## Abstract (arXiv)

Behavioural cloning uses a dataset of demonstrations to learn a behavioural policy. To overcome various learning and policy adaptation problems, we propose to use latent space to index a demonstration dataset, instantly access similar relevant experiences, and copy behavior from these situations. Actions from a selected similar situation can be performed by the agent until representations of the agent's current situation and the selected experience diverge in the latent space. Thus, we formulate our control problem as a search problem over a dataset of experts' demonstrations. We test our approach on BASALT MineRL-dataset in the latent representation of a Video PreTraining model. We compare our model to state-of-the-art Minecraft agents. Our approach can effectively recover meaningful demonstrations and show human-like behavior of an agent in the Minecraft environment in a wide variety of scenarios. Experimental results reveal that performance of our search-based approach is comparable to trained models, while allowing zero-shot task adaptation by changing the demonstration examples.