mlsanityData to 7 Oct 2026

EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation

arXiv 2406.18070 ↗First posted 26 Jun 2024cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

In this report, we present our solutions to the EgoVis Challenges in CVPR 2024, including five tracks in the Ego4D challenge and three tracks in the EPIC-Kitchens challenge. Building upon the video-language two-tower model and leveraging our meticulously organized egocentric video data, we introduce a novel foundation model called EgoVideo. This model is specifically designed to cater to the unique characteristics of egocentric videos and provides strong support for our competition submissions. In the Ego4D challenges, we tackle various tasks including Natural Language Queries, Step Grounding, Moment Queries, Short-term Object Interaction Anticipation, and Long-term Action Anticipation. In addition, we also participate in the EPIC-Kitchens challenge, where we engage in the Action Recognition, Multiple Instance Retrieval, and Domain Adaptation for Action Recognition tracks. By adapting EgoVideo to these diverse tasks, we showcase its versatility and effectiveness in different egocentric video analysis scenarios, demonstrating the powerful representation ability of EgoVideo as an egocentric foundation model. Our codebase and pretrained models are publicly available at https://github.com/OpenGVLab/EgoVideo.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

33 of 43 references.

References · 33

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2023Mistral 7Bcited by 3,065
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  10. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  11. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  12. 2020Rescaling Egocentric Visioncited by 529
21 more
  1. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  3. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  4. 2022Egocentric Video-Language Pretrainingcited by 231
  5. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  6. 2022Learning Video Representations from Large Language Modelscited by 204
  7. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  8. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  9. 2018Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transitioncited by 78
  10. 2022VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captionerscited by 69
  11. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  12. 2023NaQ: Leveraging Narrations as Queries to Supervise Episodic Memorycited by 46
  13. 2018Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systemscited by 42
  14. 2024Retrieval-Augmented Egocentric Video Captioningcited by 42
  15. 2023PALM: Predicting Actions through Language Modelscited by 28
  16. 2023StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipationcited by 23
  17. 2023GroundNLQ @ Ego4D Natural Language Queries Challenge 2023cited by 20
  18. 2023Training a Large Video Model on a Single Machine in a Daycited by 20
  19. 2023Instance Tracking in 3D Scenes from Egocentric Videoscited by 17
  20. 2023Action Sensitivity Learning for the Ego4D Episodic Memory Challenge 2023cited by 7
  21. 2018Egocentric Gesture Recognition for Head-Mounted AR devicescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 23; 2026: 14.

44 papers.

Sort:

By area:

  1. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 82
  2. 2024EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generationcited by 38
  3. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  4. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20
  5. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  6. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  7. 2025UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generationcited by 17
  8. 2025EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoTcited by 16
  9. 2024SEAL: Semantic Attention Learning for Long Video Representationcited by 12
  10. 2024Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memorycited by 11
  11. 2025Object-Shot Enhanced Grounding Network for Egocentric Videocited by 11
  12. 2025Fine-grained Spatiotemporal Grounding on Egocentric Videoscited by 11
32 more
  1. 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipationcited by 10
  2. 2025Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025cited by 9
  3. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  4. 2024VEDIT: Latent Prediction Architecture For Procedural Video Representation Learningcited by 7
  5. 2025An Egocentric Vision-Language Model based Portable Real-time Smart Assistantcited by 7
  6. 2025OSGNet @ Ego4D Episodic Memory Challenge 2025cited by 6
  7. 2025Open-ended Hierarchical Streaming Video Understanding with Vision Language Modelscited by 6
  8. 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipationcited by 5
  9. 2025Enhancing Visual Planning with Auxiliary Tasks and Multi-token Predictioncited by 5
  10. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  11. 2026MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agentscited by 5
  12. 2026FEEL (Force-Enhanced Egocentric Learning): A Dataset for Physical Action Understandingcited by 4
  13. 2024Streaming Detection of Queried Event Startcited by 3
  14. 2025EVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimizationcited by 3
  15. 2025Seeing Clearly, Forgetting Deeply: Revisiting Fine-Tuned Video Generators for Driving Simulationcited by 3
  16. 2026Optimizing Multimodal LLMs for Egocentric Video Understanding: A Solution for the HD-EPIC VQA Challengecited by 3
  17. 2026Integrating Affordances and Attention models for Short-Term Object Interaction Anticipationcited by 1
  18. 2025DIV-FF: Dynamic Image-Video Feature Fields For Environment Understanding in Egocentric Videoscited by 0
  19. 2025Reducing Annotation Burden in Physical Activity Research Using Vision-Language Modelscited by 0
  20. 2025Improving Keystep Recognition in Ego-Video via Dexterous Focuscited by 0
  21. 2025GazeNLQ @ Ego4D Natural Language Queries Challenge 2025cited by 0
  22. 2025THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Samplingcited by 0
  23. 2026Incentivizing Temporal-Awareness in Egocentric Video Understanding Modelscited by 0
  24. 2026Interactive Episodic Memory with User Feedbackcited by 0
  25. 2026EggHand: A Multimodal Foundation Model for Egocentric Hand Pose Forecastingcited by 0
  26. 2026EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understandingcited by 0
  27. 2026HiERO-StepG @ Ego4D Step Grounding Challenge: hierarchical activity understanding enables zero-shot step groundingcited by 0
  28. 2026Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Modelscited by 0
  29. 2026Towards Context-Aware Clinical Motion Understanding in Daily Living at Home: Freezing of Gait Detection with Egocentric Visioncited by 0
  30. 2026G3Ego: Gaze-Guided Graphs for Egocentric Action Understandingcited by 0
  31. 2026Linguistic Trajectory Encoding for Efficient Long-Horizon Spatial Memory in Embodied Agentscited by 0
  32. 2026Language-Augmented Video Action Anticipation: Design Fundamentals, Benchmarks, and Open Challengescited by 0