mlsanityData to 14 Oct 2026

ROI-Driven Foveated Attention for Unified Egocentric Representations in Vision-Language-Action Systems

arXiv 2603.20668 ↗First posted 21 Mar 2026cs.ROcited by 0

On the map: Vision-language-action models → Human-video and latent-action pretraining

Abstract (arXiv)

The development of embodied AI systems is increasingly constrained by the availability and structure of physical interaction data. Despite recent advances in vision-language-action (VLA) models, current pipelines suffer from high data collection cost, limited cross-embodiment alignment, and poor transfer from internet-scale visual data to robot control. We propose a region-of-interest (ROI) driven engineering workflow that introduces an egocentric, geometry-grounded data representation. By projecting end-effector poses via forward kinematics (FK) into a single external camera, we derive movement-aligned hand-centric ROIs without requiring wrist-mounted cameras or multi-view systems. Unlike directly downsampling the full frame, ROI is cropped from the original image before resizing, preserving high local information density for contact-critical regions while retaining global context. We present a reproducible pipeline covering calibration, synchronization, ROI generation, deterministic boundary handling, and metadata governance. The resulting representation is embodiment-aligned and viewpoint-normalized, enabling data reuse across heterogeneous robots. We argue that egocentric ROI serves as a practical data abstraction for scalable collection and cross-embodiment learning, bridging internet-scale perception and robot-specific control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 13 of its 20 references that are in the map. First posted on arXiv 21 Mar 2026.

Builds on

13 of 20 references.

References · 13

  1. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  3. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  5. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  8. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  9. 2014Recurrent Models of Visual Attentioncited by 349
  10. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  11. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  12. 2026SaiVLA-0: Cerebrum--Pons--Cerebellum Tripartite Architecture for Compute-Aware Vision-Language-Actioncited by 5
1 more
  1. 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  2. 2025PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
  3. 2025Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception
  4. 2026Vision-Language Models for Egocentric Video: From Hand-Object Interaction to Embodied AI
  5. 2025Developing Vision-Language-Action Model from Egocentric Videos
  6. 2025Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy
  7. 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation
  8. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗