# RoboRetriever: Single-Camera Robot Object Retrieval via Active and Interactive Perception with Dynamic Scene Graph

arXiv: https://arxiv.org/abs/2508.12916 · HTML: https://mlsanity.com/paper/2508.12916/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 18 Aug 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Built on by (most cited first, 2 of 2)

- [SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics](https://mlsanity.com/paper/2603.12193/) (2026)
- [From Passive Execution to Active Exploration: Agentic Embodied Manipulation in Realistic Environments](https://mlsanity.com/paper/2609.29091/) (2026)

## Abstract (arXiv)

Humans effortlessly retrieve objects in cluttered, partially observable environments by combining visual reasoning, active viewpoint adjustment, and physical interaction-with only a single pair of eyes. In contrast, most existing robotic systems rely on carefully positioned fixed or multi-camera setups with complete scene visibility, which limits adaptability and incurs high hardware costs. We present RoboRetriever, a novel framework for real-world object retrieval that operates using only a single wrist-mounted RGB-D camera and free-form natural language instructions. RoboRetriever grounds visual observations to build and update a dynamic hierarchical scene graph that encodes object semantics, geometry, and inter-object relations over time. The supervisor module reasons over this memory and task instruction to infer the target object and coordinate an integrated action module combining active perception, interactive perception, and manipulation. To enable task-aware scene-grounded active perception, we introduce a novel visual prompting scheme that leverages large reasoning vision-language models to determine 6-DoF camera poses aligned with the semantic task goal and geometry scene context. We evaluate RoboRetriever on diverse real-world object retrieval tasks, including scenarios with human intervention, demonstrating strong adaptability and robustness in cluttered scenes with only one RGB-D camera.