Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Humans can reproduce the viewpoint specified by a target image through active head and body motion, yet spatial intelligence in foundation models has largely been studied as passive understanding of pre-collected observations. We introduce Target Viewpoint Reproduction (TVR) -- an active task where an agent adjusts its viewpoint in a 3D environment until its observation matches a given target image -- and TVRBench, an indoor-simulation benchmark spanning scene scale and target-view visual richness. TVR is far from solved: on the evaluation split, the strongest open-source and closed-source models reach only 7.8% and 12.0% success. Fine-grained analysis identifies two consistent bottlenecks: off-the-shelf models struggle with multi-turn visual history, and performance drops sharply when viewpoint reproduction requires body translation rather than in-place rotation, exposing a gap in mapping spatial discrepancies to embodied movement. To study reducing this gap, we build a unified TVR post-training framework covering expert-trajectory SFT, rationale-supervised CoT-SFT, offline Single-turn GRPO, and on-policy Multi-turn GRPO from live simulator rollouts. Visual-action SFT supplies the main gain, raising a 9B open-source model to 50.8% success; Multi-turn GRPO provides targeted multi-room refinement and reaches 51.4% overall, while CoT supervision and Single-turn GRPO degrade closed-loop performance. These results establish TVRBench as a testbed for measuring and training foundation models that actively perceive and act in 3D environments. Our code, data, and models are available at https://github.com/aim-uofa/TVRBench.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 34 references that are in the map. First posted on arXiv 31 May 2026.
Builds on
22 of 34 references.
Closest ancestors · 4
Same direction, newest first
- 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Training1 Apr
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
- 2022SQA3D: Situated Question Answering in 3D Scenes14 Oct
Other references · 18
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
- 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 66
- 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
- 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53
6 more
- 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
- 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learningcited by 41
- 2026ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loopcited by 19
- 2026Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?cited by 18
- 2025Toward Ambulatory Vision: Learning Visually-Grounded Active View Selectioncited by 5
- 2026E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control
- 2026Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation
- 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- 2021Pathdreamer: A World Model for Indoor Navigation
- 2024AIR-Embodied: An Efficient Active 3DGS-based Interaction and Reconstruction Framework with Embodied Large Language Model
- 2026InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap