# ExploreAI: Agentic Exploration Knowledge Bases for Reproducible Observable-Regression Testing of Black-Box VR and 3D Applications

arXiv: https://arxiv.org/abs/2608.21628 · HTML: https://mlsanity.com/paper/2608.21628/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 14 of 14)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [Vision-Language Navigation: A Survey and Taxonomy](https://mlsanity.com/paper/2108.11544/) (2021)
- [NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM](https://mlsanity.com/paper/2502.11142/) (2025)
- [Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI](https://mlsanity.com/paper/2311.03783/) (2023)
- [Harnessing Large Language Model for Virtual Reality Exploration Testing: A Case Study](https://mlsanity.com/paper/2501.05625/) (2025)
- [NavAI: A Generalizable LLM Framework for Navigation Tasks in Virtual Reality Environments](https://mlsanity.com/paper/2601.03251/) (2026)
- [Automated Testing of Prevalent 3D User Interactions in Virtual Reality Applications](https://mlsanity.com/paper/2601.23139/) (2026)
- [System Test Generation for Virtual Reality Applications using Scenario Models](https://mlsanity.com/paper/2605.07534/) (2026)
- [VRExplorer: A Model-based Approach for Semi-Automated Testing of Virtual Reality Scenes](https://mlsanity.com/paper/2607.10174/) (2026)

## Abstract (arXiv)

Black-box VR and 3D applications are difficult to regression test because observable failures depend on where a tester moves, what objects are visible, and which views are captured. Manual exploratory testing can find such failures, but its evidence is time-consuming to reproduce; systematic sweeps are reproducible, but they lack semantic guidance and spend exploration budget on low-value viewpoints. We observe that an LLM can make the high-level decisions a human tester makes during exploration: interpreting a task, choosing which objects to inspect, grouping related objects, recording what it saw, and deciding when missing evidence should trigger another attempt. Based on this observation, we present ExploreAI, an LLM-driven agentic framework that offloads repeated perception, navigation, multi-view capture execution, and logging to specialized modules while using the LLM for planning, evidence recording, capture-policy decisions, and verification decisions. ExploreAI constructs an Exploration Knowledge Base (EKB): a structured, per-object record of one exploration run. For each object the agent finds, the EKB stores the scan evidence that exposed it, the selected target, the navigation path, the multi-view capture, and the self-verification result. The EKB is a reusable testing artifact that supports reproducible observable-regression checking across versions of a VR or 3D application. Across six indoor and outdoor scenes in Unity, AI2-THOR, and BeamNG, ExploreAI constructs high-completeness EKBs under both complete and target exploration, and an LLM-module ablation shows where semantic planning, capture policy, evidence recording, and self-verification contribute. Reproduction pilots further show that EKB-guided traces help both humans and LLM-based reproducers reproduce exact object-view evidence more effectively than conditions without EKB context.