# VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Games

arXiv: https://arxiv.org/abs/2511.00002 · HTML: https://mlsanity.com/paper/2511.00002/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [Imitation Learning: Progress, Taxonomies and Challenges](https://mlsanity.com/paper/2106.12177/) (2021)
- [Counter-Strike Deathmatch with Large-Scale Behavioural Cloning](https://mlsanity.com/paper/2104.04258/) (2021)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformer](https://mlsanity.com/paper/2401.17698/) (2024)
- [The Pitfalls of Imitation Learning when Actions are Continuous](https://mlsanity.com/paper/2503.09722/) (2025)
- [Benchmarking End-to-End Behavioural Cloning on Video Games](https://mlsanity.com/paper/2004.00981/) (2020)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games](https://mlsanity.com/paper/2307.11105/) (2023)
- [Playing Minecraft with Behavioural Cloning](https://mlsanity.com/paper/2005.03374/) (2020)
- [VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research](https://mlsanity.com/paper/2306.03381/) (2023)
- [Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games](https://mlsanity.com/paper/2312.02312/) (2023)
- [Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment](https://mlsanity.com/paper/2408.08396/) (2024)
- [Human-AI Collaborative Game Testing with Vision Language Models](https://mlsanity.com/paper/2501.11782/) (2025)

## Abstract (arXiv)

Virtual Reality (VR) has rapidly become a mainstream platform for gaming and interactive experiences, yet ensuring the quality, safety, and appropriateness of VR content remains a pressing challenge. Traditional human-based quality assurance is labor-intensive and cannot scale with the industry's rapid growth. While automated testing has been applied to traditional 2D and 3D games, extending it to VR introduces unique difficulties due to high-dimensional sensory inputs and strict real-time performance requirements. We present VRScout, a deep learning-based agent capable of autonomously navigating VR environments and interacting with virtual objects in a human-like and real-time manner. VRScout learns from human demonstrations using an enhanced Action Chunking Transformer that predicts multi-step action sequences. This enables our agent to capture higher-level strategies and generalize across diverse environments. To balance responsiveness and precision, we introduce a dynamically adjustable sliding horizon that adapts the agent's temporal context at runtime. We evaluate VRScout on commercial VR titles and show that it achieves expert-level performance with only limited training data, while maintaining real-time inference at 60 FPS on consumer-grade hardware. These results position VRScout as a practical and scalable framework for automated VR game testing, with direct applications in both quality assurance and safety auditing.