# Vision in Action: Learning Active Perception from Human Demonstrations

arXiv: https://arxiv.org/abs/2506.15666 · HTML: https://mlsanity.com/paper/2506.15666/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2025.

## Built on by (most cited first, 30 of 37)

- [EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World](https://mlsanity.com/paper/2604.07607/) (2026)
- [TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System](https://mlsanity.com/paper/2511.02832/) (2025)
- [Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections](https://mlsanity.com/paper/2506.16685/) (2025)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations](https://mlsanity.com/paper/2510.01607/) (2025)
- [OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction](https://mlsanity.com/paper/2512.16842/) (2025)
- [OAT: Ordered Action Tokenization](https://mlsanity.com/paper/2602.04215/) (2026)
- [SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics](https://mlsanity.com/paper/2603.12193/) (2026)
- [ActiveMimic: Egocentric Video Pretraining with Active Perception](https://mlsanity.com/paper/2606.06194/) (2026)
- [Towards Exploratory and Focused Manipulation with Bimanual Active Perception: A New Problem, Benchmark and Strategy](https://mlsanity.com/paper/2602.01939/) (2026)
- [SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models](https://mlsanity.com/paper/2602.04208/) (2026)
- [Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action](https://mlsanity.com/paper/2605.22283/) (2026)
- [E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2604.17969/) (2026)
- [TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning](https://mlsanity.com/paper/2605.07943/) (2026)
- [Observer-Actor: Active Vision Imitation Learning with Sparse-View Gaussian Splatting](https://mlsanity.com/paper/2511.18140/) (2025)
- [Mobile UMI: Cross-View Diffusion Policy with Decoupled Kinematics for Mobile Manipulation](https://mlsanity.com/paper/2605.20894/) (2026)
- [LIME: Learning Intent-aware Camera Motion from Egocentric Video](https://mlsanity.com/paper/2607.02417/) (2026)
- [Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning](https://mlsanity.com/paper/2510.08442/) (2025)
- [Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception](https://mlsanity.com/paper/2511.15279/) (2025)
- [Real-World Reinforcement Learning of Active Perception Behaviors](https://mlsanity.com/paper/2512.01188/) (2025)
- [Tac-DINO: Learning Vision-Tactile Features with Patch Alignment](https://mlsanity.com/paper/2606.12069/) (2026)
- [See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation](https://mlsanity.com/paper/2606.13279/) (2026)
- [Learning to See While Learning to Act: Diffusion Models for Active Perception in Robot Imitation](https://mlsanity.com/paper/2606.23625/) (2026)
- [VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control](https://mlsanity.com/paper/2609.19554/) (2026)
- [Active Stereo-Camera Outperforms Multi-Sensor Setup in ACT Imitation Learning for Humanoid Manipulation](https://mlsanity.com/paper/2603.28422/) (2026)
- [The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection](https://mlsanity.com/paper/2607.02322/) (2026)
- [ModPack: Extensible Teleoperation Interface for Bimanual Mobile Manipulation](https://mlsanity.com/paper/2607.19479/) (2026)
- [ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardware](https://mlsanity.com/paper/2609.18514/) (2026)
- [HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction](https://mlsanity.com/paper/2609.18548/) (2026)
- [INSPECT: Learning Robot View Selection from Assistant Use](https://mlsanity.com/paper/2609.20615/) (2026)

## Abstract (arXiv)

We present Vision in Action (ViA), an active perception system for bimanual robot manipulation. ViA learns task-relevant active perceptual strategies (e.g., searching, tracking, and focusing) directly from human demonstrations. On the hardware side, ViA employs a simple yet effective 6-DoF robotic neck to enable flexible, human-like head movements. To capture human active perception strategies, we design a VR-based teleoperation interface that creates a shared observation space between the robot and the human operator. To mitigate VR motion sickness caused by latency in the robot's physical movements, the interface uses an intermediate 3D scene representation, enabling real-time view rendering on the operator side while asynchronously updating the scene with the robot's latest observations. Together, these design elements enable the learning of robust visuomotor policies for three complex, multi-stage bimanual manipulation tasks involving visual occlusions, significantly outperforming baseline systems.