# GaussAnything: Semantic Intent-Driven Refinement of Evolving Gaussian Scenes for Standalone VR

arXiv: https://arxiv.org/abs/2609.13859 · HTML: https://mlsanity.com/paper/2609.13859/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2026.

## Built on by (most cited first, 1 of 1)

- [CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation](https://mlsanity.com/paper/2609.31418/) (2026)

## Abstract (arXiv)

Deploying reconstructed 3D environments on standalone VR headsets are constrained by limited compute and memory, and conventional level-of-detail policies optimize for visibility without accounting for the user's explicit inspection intent. We present GaussAnything, a native OpenXR system for intent-conditioned reallocation and progressive publication of evolving semantic Gaussian+SDF scenes. GaussAnything resolves class- or instance-level queries to persistent 3D objects and reallocates a fixed Gaussian resident budget toward the selected object while retaining global context, applying incremental, stable-identity updates coordinated with the TSDF-derived mesh through a source-epoch mechanism. Across eight scenes, an object query concentrates 88-90% of the fixed client budget onto the queried object without enlarging it, on-device rendering reproduces the host render to within a small margin (up to 36.7 dB), and the standalone client renders each stereo frame at a steady-state GPU cost of roughly 10 ms within the frame budget of standard standalone panels.