mlsanityData to 14 Oct 2026

Ego4D: Around the World in 3,000 Hours of Egocentric Video

arXiv 2110.07058 ↗First posted 13 Oct 2021cs.CV cs.AIcited by 1,556

Not on the map. Why?

Cited by 196 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception. Project page: https://ego4d-data.org/

Key facts. Cited by 1,556 papers in the map (data through 14 Oct 2026). Builds on 131 of its 228 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

131 of 228 references.

References · 131

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2017Mask R-CNNcited by 7,277
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  12. 2015Fast R-CNNcited by 2,933
38 more
  1. 2018YOLOv3: An Incremental Improvementcited by 2,659
  2. 2015VQA: Visual Question Answeringcited by 2,506
  3. 2017Non-local Neural Networkscited by 2,293
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  5. 2017The Kinetics Human Action Video Datasetcited by 1,909
  6. 2018SlowFast Networks for Video Recognitioncited by 1,867
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  8. 2021Vision Transformers for Dense Predictioncited by 1,803
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  10. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  12. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  13. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  14. 2019Objects as Pointscited by 1,175
  15. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  16. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  17. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  18. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  19. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  20. 2021Multiscale Vision Transformerscited by 897
  21. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  22. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  23. 2017Dense-Captioning Events in Videoscited by 793
  24. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  25. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  26. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  27. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  28. 2017Deep Layer Aggregationcited by 436
  29. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429
  30. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  31. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  32. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 343
  33. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  34. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 329
  35. 2017Temporal Relational Reasoning in Videoscited by 324
  36. 2018Deep Audio-Visual Speech Recognitioncited by 322
  37. 2015Deep multi-scale video prediction beyond mean square errorcited by 300
  38. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285

50 most cited of 131 All 131 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 111; 2023: 236; 2024: 341; 2025: 429; 2026: 433.

1,556 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,234
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  4. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  5. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  6. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  8. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  9. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  10. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  11. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  12. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
38 more
  1. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  2. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  3. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  4. 2026World Action Models are Zero-shot Policiescited by 465
  5. 2023Learning Interactive Real-World Simulatorscited by 458
  6. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
  7. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  8. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  9. 2025Kimi-VL Technical Reportcited by 388
  10. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  11. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
  12. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  13. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  14. 2024Latent Action Pretraining from Videoscited by 321
  15. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  16. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 314
  17. 2023Reconstructing Hands in 3D with Transformerscited by 306
  18. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  19. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  20. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  21. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  22. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  23. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  24. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
  25. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  26. 2024Navigation World Modelscited by 272
  27. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 267
  28. 2023A Cookbook of Self-Supervised Learningcited by 250
  29. 2022Masked Visual Pre-training for Motor Controlcited by 248
  30. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  31. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  32. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  33. 2022Egocentric Video-Language Pretrainingcited by 232
  34. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  35. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  36. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
  37. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  38. 2023ViNT: A Foundation Model for Visual Navigationcited by 222

50 most cited of 1,556 All 1,556 (JSON)