mlsanityData to 7 Oct 2026

An Outlook into the Future of Egocentric Vision

arXiv 2308.07123 ↗First posted 14 Aug 2023cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

What will the future be? We wonder! In this survey, we explore the gap between current research in egocentric vision and the ever-anticipated future, where wearable computing, with outward facing cameras and digital overlays, is expected to be integrated in our every day lives. To understand this gap, the article starts by envisaging the future through character-based stories, showcasing through examples the limitations of current technology. We then provide a mapping between this future and previously defined research tasks. For each task, we survey its seminal works, current state-of-the-art methodologies and available datasets, then reflect on shortcomings that limit its applicability to future research. Note that this survey focuses on software models for egocentric vision, independent of any specific hardware. The paper concludes with recommendations for areas of immediate explorations so as to unlock our path to the future always-on, personalised and life-enhancing egocentric vision.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 201 of its 401 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

201 of 401 references.

References · 201

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2021ViViT: A Video Vision Transformercited by 1,610
  3. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  5. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 935
  8. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  9. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  11. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  12. 2020Rescaling Egocentric Visioncited by 529
38 more
  1. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  2. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  3. 2017Embodied Question Answeringcited by 335
  4. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  5. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  6. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  7. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  8. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  9. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  10. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  11. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  12. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  13. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  14. 2022Learning Video Representations from Large Language Modelscited by 204
  15. 2021Anticipative Video Transformercited by 197
  16. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  17. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  18. 2022Rethinking Visual Geo-localization for Large-Scale Applicationscited by 160
  19. 2023MixVPR: Feature Mixing for Visual Place Recognitioncited by 160
  20. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 157
  21. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  22. 2022Multiview Transformers for Video Recognitioncited by 148
  23. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  24. 2016Video Summarization with Long Short-term Memorycited by 142
  25. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  26. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  27. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  28. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  29. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  30. 2020Multi-Modal Domain Adaptation for Fine-Grained Action Recognitioncited by 115
  31. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  32. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  33. 2021Learning Multi-Scene Absolute Pose Regression with Transformerscited by 108
  34. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
  35. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
  36. 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
  37. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 102
  38. 2023R2Former: Unified Retrieval and Reranking Transformer for Place Recognitioncited by 102

50 most cited of 201 All 201 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 18; 2025: 35; 2026: 27.

84 papers.

Sort:

By area:

  1. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  2. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  3. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  4. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
  5. 2023Learning Object State Changes in Videos: An Open-World Perspectivecited by 36
  6. 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 36
  7. 2024EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understandingcited by 35
  8. 2024EgoLM: Multi-Modal Language Model of Egocentric Motionscited by 28
  9. 2024Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videoscited by 24
  10. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24
  11. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20
  12. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
38 more
  1. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  2. 2025EgoBlind: Towards Egocentric Visual Assistance for the Blindcited by 19
  3. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  4. 2024AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipationcited by 14
  5. 2025X-LeBench: A Benchmark for Extremely Long Egocentric Video Understandingcited by 14
  6. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  7. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  8. 2024Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions?cited by 11
  9. 2024Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memorycited by 11
  10. 2025Fine-grained Spatiotemporal Grounding on Egocentric Videoscited by 11
  11. 2025Developing Vision-Language-Action Model from Egocentric Videoscited by 10
  12. 2024A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectivescited by 9
  13. 2025MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generationcited by 9
  14. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  15. 2025MICA: Multi-Agent Industrial Coordination Assistantcited by 8
  16. 2025An Egocentric Vision-Language Model based Portable Real-time Smart Assistantcited by 7
  17. 2025EgoControl: Controllable Egocentric Video Generation via 3D Full-Body Posescited by 7
  18. 2024Bidirectional Progressive Transformer for Interaction Intention Anticipationcited by 6
  19. 2024Egocentric zone-aware action recognition across environmentscited by 6
  20. 2025ToG-Bench: Task-Oriented Spatio-Temporal Grounding in Egocentric Videoscited by 6
  21. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 6
  22. 2025Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectivescited by 5
  23. 2025EgoDTM: Towards 3D-Aware Egocentric Video-Language Pretrainingcited by 5
  24. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  25. 2025EgoPrivacy: What Your First-Person Camera Says About You?cited by 4
  26. 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understandingcited by 4
  27. 2025EgoEvGesture: Gesture Recognition Based on Egocentric Event Cameracited by 3
  28. 2025Seeing Clearly, Forgetting Deeply: Revisiting Fine-Tuned Video Generators for Driving Simulationcited by 3
  29. 2025Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistancecited by 3
  30. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 3
  31. 2025Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videoscited by 2
  32. 2025Zero-Shot Temporal Interaction Localization for Egocentric Videoscited by 2
  33. 2025HanDyVQA: A Video QA Benchmark for Fine-Grained Hand-Object Interaction Dynamicscited by 2
  34. 2026EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VRcited by 2
  35. 2026ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarioscited by 2
  36. 2026Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videoscited by 2
  37. 2026Vinci2: Providing Proactive Assistance in Continuous Egocentric Videoscited by 2
  38. 2024On the Application of Egocentric Computer Vision to Industrial Scenarioscited by 1

50 most cited of 84 All 84 (JSON)