# AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systems

arXiv: https://arxiv.org/abs/2409.07189 · HTML: https://mlsanity.com/paper/2409.07189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 75 references that are in the map.
- First posted on arXiv 11 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [The NetHack Learning Environment](https://mlsanity.com/paper/2006.13760/) (2020)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Fighting Copycat Agents in Behavioral Cloning from Observation Histories](https://mlsanity.com/paper/2010.14876/) (2020)
- [Feedback in Imitation Learning: The Three Regimes of Covariate Shift](https://mlsanity.com/paper/2102.02872/) (2021)
- [Imitation Learning: Progress, Taxonomies and Challenges](https://mlsanity.com/paper/2106.12177/) (2021)
- [Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations](https://mlsanity.com/paper/2402.14606/) (2024)
- [Causal Imitation Learning under Temporally Correlated Noise](https://mlsanity.com/paper/2202.01312/) (2022)
- [Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction](https://mlsanity.com/paper/2207.03395/) (2022)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction](https://mlsanity.com/paper/2207.09705/) (2022)
- [A Survey of Demonstration Learning](https://mlsanity.com/paper/2303.11191/) (2023)
- [Scaling Laws for Imitation Learning in Single-Agent Games](https://mlsanity.com/paper/2307.09423/) (2023)
- [Autonomous robotic nanofabrication with reinforcement learning](https://mlsanity.com/paper/2002.11952/) (2020)
- [Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments](https://mlsanity.com/paper/2209.09233/) (2022)
- [ARC - Actor Residual Critic for Adversarial Imitation Learning](https://mlsanity.com/paper/2206.02095/) (2022)
- [Interactive molecular dynamics in virtual reality from quantum chemistry to drug binding: An open-source multi-person framework](https://mlsanity.com/paper/1902.01827/) (2019)
- [Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learning](https://mlsanity.com/paper/2310.06801/) (2023)
- [MAAIP: Multi-Agent Adversarial Interaction Priors for imitation from fighting demonstrations for physics-based characters](https://mlsanity.com/paper/2311.02502/) (2023)
- [A Domain Specific Language for Performance Portable Molecular Dynamics Algorithms](https://mlsanity.com/paper/1704.03329/) (2017)

## Abstract (arXiv)

Molecular dynamics (MD) simulations are a crucial computational tool for researchers to understand and engineer molecular structure and function in areas such as drug discovery, protein engineering, and material design. Despite their utility, MD simulations are expensive, owing to the high dimensionality of molecular systems. Interactive molecular dynamics in virtual reality (iMD-VR) has recently emerged as a "human-in-the-loop" strategy for efficiently navigating hyper-dimensional molecular systems. By providing an immersive 3D environment that enables visualization and manipulation of real-time molecular simulations running on high-performance computing architectures, iMD-VR enables researchers to reach out and guide molecular conformational dynamics, in order to efficiently explore complex, high-dimensional molecular systems. Moreover, iMD-VR simulations generate rich datasets that capture human experts' spatial insight regarding molecular structure and function. This paper explores the use of researcher-generated iMD-VR datasets to train AI agents via imitation learning (IL). IL enables agents to mimic complex behaviours from expert demonstrations, circumventing the need for explicit programming or intricate reward design. In this article, we review IL across robotics and Multi-agents systems domains which are comparable to iMD-VR, and discuss how iMD-VR recordings could be used to train IL models to interact with MD simulations. We then illustrate the applications of these ideas through a proof-of-principle study where iMD-VR data was used to train a CNN network on a simple molecular manipulation task; namely, threading a small molecule through a nanotube pore. Finally, we outline future research directions and potential challenges of using AI agents to augment human expertise in navigating vast molecular conformational spaces.