# RoboPanoptes: The All-seeing Robot with Whole-body Dexterity

arXiv: https://arxiv.org/abs/2501.05420 · HTML: https://mlsanity.com/paper/2501.05420/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2025.

## Built on by (most cited first, 9 of 9)

- [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025)
- [BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities](https://mlsanity.com/paper/2503.05652/) (2025)
- [Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections](https://mlsanity.com/paper/2506.16685/) (2025)
- [Masked Visual Actions for Unified World Modeling](https://mlsanity.com/paper/2607.19343/) (2026)
- [3D Equivariant Visuomotor Policy Learning via Spherical Projection](https://mlsanity.com/paper/2505.16969/) (2025)
- [Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperation](https://mlsanity.com/paper/2608.02257/) (2026)
- [Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation](https://mlsanity.com/paper/2605.20894/) (2026)
- [Learning to Act under Visual Interruptions with Vision-Language-Action Models](https://mlsanity.com/paper/2609.35003/) (2026)
- [Blackout vs. Freeze: Analyzing Physical Failure Modes of VLAs under Camera Faults](https://mlsanity.com/paper/2609.39145/) (2026)

## Abstract (arXiv)

We present RoboPanoptes, a capable yet practical robot system that achieves whole-body dexterity through whole-body vision. Its whole-body dexterity allows the robot to utilize its entire body surface for manipulation, such as leveraging multiple contact points or navigating constrained spaces. Meanwhile, whole-body vision uses a camera system distributed over the robot's surface to provide comprehensive, multi-perspective visual feedback of its own and the environment's state. At its core, RoboPanoptes uses a whole-body visuomotor policy that learns complex manipulation skills directly from human demonstrations, efficiently aggregating information from the distributed cameras while maintaining resilience to sensor failures. Together, these design aspects unlock new capabilities and tasks, allowing RoboPanoptes to unbox in narrow spaces, sweep multiple or oversized objects, and succeed in multi-step stowing in cluttered environments, outperforming baselines in adaptability and efficiency. Results are best viewed on https://robopanoptes.github.io.