# EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding

arXiv: https://arxiv.org/abs/2301.02217 · HTML: https://mlsanity.com/paper/2301.02217/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 72 references that are in the map.
- First posted on arXiv 5 Jan 2023.

## Builds on (most cited first, 30 of 53)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization](https://mlsanity.com/paper/1807.00230/) (2018)
- [SoundNet: Learning Sound Representations from Unlabeled Video](https://mlsanity.com/paper/1610.09001/) (2016)
- [EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition](https://mlsanity.com/paper/1908.08498/) (2019)
- [Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers](https://mlsanity.com/paper/2106.05392/) (2021)
- [Omnivore: A Single Model for Many Visual Modalities](https://mlsanity.com/paper/2201.08377/) (2022)
- [Cross Modal Distillation for Supervision Transfer](https://mlsanity.com/paper/1507.00448/) (2015)
- [Listen to Look: Action Recognition by Previewing Audio](https://mlsanity.com/paper/1912.04487/) (2019)
- [What Makes Multi-modal Learning Better than Single (Provably)](https://mlsanity.com/paper/2106.04538/) (2021)
- [MoViNets: Mobile Video Networks for Efficient Video Recognition](https://mlsanity.com/paper/2103.11511/) (2021)
- [SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition](https://mlsanity.com/paper/1904.04289/) (2019)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos](https://mlsanity.com/paper/1804.09626/) (2018)
- [ECO: Efficient Convolutional Network for Online Video Understanding](https://mlsanity.com/paper/1804.09066/) (2018)
- [Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos](https://mlsanity.com/paper/2204.01696/) (2022)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [AR-Net: Adaptive Frame Resolution for Efficient Action Recognition](https://mlsanity.com/paper/2007.15796/) (2020)
- [Modality Distillation with Multiple Stream Networks for Action Recognition](https://mlsanity.com/paper/1806.07110/) (2018)

## Built on by (most cited first, 30 of 32)

- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans](https://mlsanity.com/paper/2312.00775/) (2023)
- [EPIC Fields: Marrying 3D Geometry and Video Understanding](https://mlsanity.com/paper/2306.08731/) (2023)
- [Multimodal Distillation for Egocentric Action Recognition](https://mlsanity.com/paper/2307.07483/) (2023)
- [EgoLM: Multi-Modal Language Model of Egocentric Motions](https://mlsanity.com/paper/2409.18127/) (2024)
- [EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Views](https://mlsanity.com/paper/2405.13659/) (2024)
- [UniMTS: Unified Pre-training for Motion Time Series](https://mlsanity.com/paper/2410.19818/) (2024)
- [Challenges and Trends in Egocentric Vision: A Survey](https://mlsanity.com/paper/2503.15275/) (2025)
- [Wearable Accelerometer Foundation Models for Health via Knowledge Distillation](https://mlsanity.com/paper/2412.11276/) (2024)
- [Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition](https://mlsanity.com/paper/2405.19917/) (2024)
- [Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenes](https://mlsanity.com/paper/2311.17948/) (2023)
- [Self-supervised visual learning from interactions with objects](https://mlsanity.com/paper/2407.06704/) (2024)
- [Multi-modal Knowledge Distillation-based Human Trajectory Forecasting](https://mlsanity.com/paper/2503.22201/) (2025)
- [COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition](https://mlsanity.com/paper/2503.07259/) (2025)
- [EgoControl: Controllable Egocentric Video Generation via 3D Full-Body Poses](https://mlsanity.com/paper/2511.18173/) (2025)
- [EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses](https://mlsanity.com/paper/2508.01915/) (2025)
- [Spherical World-Locking for Audio-Visual Localization in Egocentric Videos](https://mlsanity.com/paper/2408.05364/) (2024)
- [HANDI: Hand-Centric Text-and-Image Conditioned Video Generation](https://mlsanity.com/paper/2412.04189/) (2024)
- [EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception](https://mlsanity.com/paper/2506.21080/) (2025)
- [EgoPrompt: Prompt Learning for Egocentric Action Recognition](https://mlsanity.com/paper/2508.03266/) (2025)
- [SkillSight: Efficient First-Person Skill Assessment with Gaze](https://mlsanity.com/paper/2511.19629/) (2025)
- [Seeing without Pixels: Perception from Camera Trajectories](https://mlsanity.com/paper/2511.21681/) (2025)
- [Learning Predictive Visuomotor Coordination](https://mlsanity.com/paper/2503.23300/) (2025)
- [Multimodal Knowledge Distillation for Egocentric Action Recognition Robust to Missing Modalities](https://mlsanity.com/paper/2504.08578/) (2025)
- [Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffusers](https://mlsanity.com/paper/2507.11287/) (2025)
- [SneakPeek: Future-Guided Instructional Streaming Video Generation](https://mlsanity.com/paper/2512.13019/) (2025)
- [MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignment](https://mlsanity.com/paper/2602.19004/) (2026)
- [How You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Prediction](https://mlsanity.com/paper/2605.20388/) (2026)
- [GazeNLQ @ Ego4D Natural Language Queries Challenge 2025](https://mlsanity.com/paper/2506.05782/) (2025)
- [THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling](https://mlsanity.com/paper/2507.06442/) (2025)

## Abstract (arXiv)

Recent advances in egocentric video understanding models are promising, but their heavy computational expense is a barrier for many real-world applications. To address this challenge, we propose EgoDistill, a distillation-based approach that learns to reconstruct heavy egocentric video clip features by combining the semantics from a sparse set of video frames with the head motion from lightweight IMU readings. We further devise a novel self-supervised training strategy for IMU feature learning. Our method leads to significant improvements in efficiency, requiring 200x fewer GFLOPs than equivalent video models. We demonstrate its effectiveness on the Ego4D and EPICKitchens datasets, where our method outperforms state-of-the-art efficient video understanding methods.