# Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation

arXiv: https://arxiv.org/abs/2102.01295 · HTML: https://mlsanity.com/paper/2102.01295/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Deep Spatial Autoencoders for Visuomotor Learning](https://mlsanity.com/paper/1509.06113/) (2015)
- [Deep Reinforcement Learning for High Precision Assembly Tasks](https://mlsanity.com/paper/1708.04033/) (2017)
- [AGIL: Learning Attention from Human for Visuomotor Tasks](https://mlsanity.com/paper/1806.03960/) (2018)
- [A Laser-based Dual-arm System for Precise Control of Collaborative Robots](https://mlsanity.com/paper/2011.01573/) (2020)

## Built on by (most cited first, 17 of 17)

- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [Transformer-based deep imitation learning for dual-arm robot manipulation](https://mlsanity.com/paper/2108.00385/) (2021)
- [Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation](https://mlsanity.com/paper/2203.09749/) (2022)
- [SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering](https://mlsanity.com/paper/2210.15185/) (2022)
- [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://mlsanity.com/paper/2507.15833/) (2025)
- [Memory-based gaze prediction in deep imitation learning for robot manipulation](https://mlsanity.com/paper/2202.04877/) (2022)
- [Training Robots without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer](https://mlsanity.com/paper/2202.09574/) (2022)
- [Gaze-Regularized Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2603.23202/) (2026)
- [Multi-task real-robot data with gaze attention for dual-arm fine manipulation](https://mlsanity.com/paper/2401.07603/) (2024)
- [Imitation Learning with Human Eye Gaze via Multi-Objective Prediction](https://mlsanity.com/paper/2102.13008/) (2021)
- [Generalizable Imitation Learning Through Pre-Trained Representations](https://mlsanity.com/paper/2311.09350/) (2023)
- [Gaze-Guided Task Decomposition for Imitation Learning in Robotic Manipulation](https://mlsanity.com/paper/2501.15071/) (2025)
- [Enhancing Reusability of Learned Skills for Robot Manipulation via Gaze Information and Motion Bottlenecks](https://mlsanity.com/paper/2502.18121/) (2025)
- [Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach](https://mlsanity.com/paper/2605.07166/) (2026)
- [GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction](https://mlsanity.com/paper/2609.38519/) (2026)
- [FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs](https://mlsanity.com/paper/2610.20064/) (2026)

## Abstract (arXiv)

A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency. Data from this and related works are available at: https://sites.google.com/view/multi-task-fine.