# Real-time Pupil Tracking from Monocular Video for Digital Puppetry

arXiv: https://arxiv.org/abs/2006.11341 · HTML: https://mlsanity.com/paper/2006.11341/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorders](https://mlsanity.com/paper/2309.10077/) (2023)
- [DOOMGAN:High-Fidelity Dynamic Identity Obfuscation Ocular Generative Morphing](https://mlsanity.com/paper/2507.17158/) (2025)
- [AutoPtosis](https://mlsanity.com/paper/2106.03905/) (2021)
- [Fast Eye Detector Using Siamese Network for NIR Partial Face Images](https://mlsanity.com/paper/2202.10671/) (2022)
- [APPLE: Attribute-Preserving Pseudo-Labeling for Diffusion-Based Face Swapping](https://mlsanity.com/paper/2601.15288/) (2026)

## Abstract (arXiv)

We present a simple, real-time approach for pupil tracking from live video on mobile devices. Our method extends a state-of-the-art face mesh detector with two new components: a tiny neural network that predicts positions of the pupils in 2D, and a displacement-based estimation of the pupil blend shape coefficients. Our technique can be used to accurately control the pupil movements of a virtual puppet, and lends liveliness and energy to it. The proposed approach runs at over 50 FPS on modern phones, and enables its usage in any real-time puppeteering pipeline.