mlsanityData to 7 Oct 2026

EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding

arXiv 2301.02217 ↗First posted 5 Jan 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Recent advances in egocentric video understanding models are promising, but their heavy computational expense is a barrier for many real-world applications. To address this challenge, we propose EgoDistill, a distillation-based approach that learns to reconstruct heavy egocentric video clip features by combining the semantics from a sparse set of video frames with the head motion from lightweight IMU readings. We further devise a novel self-supervised training strategy for IMU feature learning. Our method leads to significant improvements in efficiency, requiring 200x fewer GFLOPs than equivalent video models. We demonstrate its effectiveness on the Ego4D and EPICKitchens datasets, where our method outperforms state-of-the-art efficient video understanding methods.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 53 of its 72 references that are in the map. First posted on arXiv 5 Jan 2023.

Builds on

53 of 72 references.

References · 53

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2018SlowFast Networks for Video Recognitioncited by 1,862
  5. 2017Decoupled Weight Decay Regularizationcited by 1,722
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  8. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  9. 2021Multiscale Vision Transformerscited by 896
  10. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  11. 2020Rescaling Egocentric Visioncited by 529
  12. 2017Look, Listen and Learncited by 452
38 more
  1. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  2. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 246
  3. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
  4. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  5. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  6. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  7. 2015Cross Modal Distillation for Supervision Transfercited by 173
  8. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  9. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  10. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  11. 2017Deep Learning for Sensor-based Activity Recognition: A Surveycited by 133
  12. 2019SCSampler: Sampling Salient Clips from Video for Efficient Action Recognitioncited by 133
  13. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
  14. 2018ECO: Efficient Convolutional Network for Online Video Understandingcited by 115
  15. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  16. 2015On-Manifold Preintegration for Real-Time Visual-Inertial Odometrycited by 95
  17. 2020AR-Net: Adaptive Frame Resolution for Efficient Action Recognitioncited by 88
  18. 2018Modality Distillation with Multiple Stream Networks for Action Recognitioncited by 83
  19. 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
  20. 2021Adaptive Focus for Efficient Video Recognitioncited by 76
  21. 2019RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methodscited by 72
  22. 2019Self-supervised Moving Vehicle Tracking with Stereo Soundcited by 63
  23. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  24. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
  25. 2021E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognitioncited by 50
  26. 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
  27. 20213D-to-2D Distillation for Indoor Scene Parsingcited by 47
  28. 2015Learning image representations tied to ego-motioncited by 41
  29. 2017RIDI: Robust IMU Double Integrationcited by 41
  30. 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognitioncited by 37
  31. 2021AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognitioncited by 36
  32. 2022Episodic Memory Question Answeringcited by 32
  33. 2022Egocentric Prediction of Action Target in 3Dcited by 31
  34. 2020Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimationcited by 26
  35. 2018Who Let The Dogs Out? Modeling Dog Behavior From Visual Datacited by 20
  36. 2022Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selectioncited by 17
  37. 2021RIO: Rotation-equivariance supervised learning of robust inertial odometrycited by 14
  38. 2020Vision and Inertial Sensing Fusion for Human Action Recognition : A Reviewcited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 15; 2026: 4.

32 papers.

Sort:

By area:

  1. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  2. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  3. 2023EPIC Fields: Marrying 3D Geometry and Video Understandingcited by 49
  4. 2023Multimodal Distillation for Egocentric Action Recognitioncited by 28
  5. 2024EgoLM: Multi-Modal Language Model of Egocentric Motionscited by 28
  6. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  7. 2024UniMTS: Unified Pre-training for Motion Time Seriescited by 22
  8. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  9. 2024Wearable Accelerometer Foundation Models for Health via Knowledge Distillationcited by 15
  10. 2024Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognitioncited by 14
  11. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
  12. 2024Self-supervised visual learning from interactions with objectscited by 10
20 more
  1. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  2. 2025COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognitioncited by 8
  3. 2025EgoControl: Controllable Egocentric Video Generation via 3D Full-Body Posescited by 7
  4. 2025EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glassescited by 6
  5. 2024Spherical World-Locking for Audio-Visual Localization in Egocentric Videoscited by 5
  6. 2024HANDI: Hand-Centric Text-and-Image Conditioned Video Generationcited by 5
  7. 2025EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perceptioncited by 4
  8. 2025EgoPrompt: Prompt Learning for Egocentric Action Recognitioncited by 4
  9. 2025SkillSight: Efficient First-Person Skill Assessment with Gazecited by 4
  10. 2025Seeing without Pixels: Perception from Camera Trajectoriescited by 4
  11. 2025Learning Predictive Visuomotor Coordinationcited by 2
  12. 2025Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffuserscited by 1
  13. 2025SneakPeek: Future-Guided Instructional Streaming Video Generationcited by 1
  14. 2026MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignmentcited by 1
  15. 2026How You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Predictioncited by 1
  16. 2025Multimodal Knowledge Distillation for Egocentric Action Recognition Robust to Missing Modalitiescited by 0
  17. 2025GazeNLQ @ Ego4D Natural Language Queries Challenge 2025cited by 0
  18. 2025THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Samplingcited by 0
  19. 2026Beyond MACs: Hardware Efficient Architecture Design for Vision Backbonescited by 0
  20. 2026UNIEGO: Proxies as Mediators for Unified Egocentric Video Representation Learningcited by 0