mlsanityData to 7 Oct 2026

Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos

arXiv 1804.09626 ↗First posted 25 Apr 2018cs.CVcited by 127

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In Actor and Observer we introduced a dataset linking the first and third-person video understanding domains, the Charades-Ego Dataset. In this paper we describe the egocentric aspect of the dataset and present annotations for Charades-Ego with 68,536 activity instances in 68.8 hours of first and third-person video, making it one of the largest and most diverse egocentric datasets available. Charades-Ego furthermore shares activity classes, scripts, and methodology with the Charades dataset, that consist of additional 82.3 hours of third-person video with 66,500 activity instances. Charades-Ego has temporal annotations and textual descriptions, making it suitable for egocentric video classification, localization, captioning, and new tasks utilizing the cross-modal nature of the data.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 15; 2023: 29; 2024: 28; 2025: 24; 2026: 19.

127 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  3. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  4. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  5. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  6. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  7. 2022Egocentric Video-Language Pretrainingcited by 231
  8. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  9. 2022Learning Video Representations from Large Language Modelscited by 204
  10. 2023Video Understanding with Large Language Models: A Surveycited by 203
  11. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  12. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
38 more
  1. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  2. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  3. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
  4. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  5. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  6. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  7. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  8. 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planningcited by 66
  9. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  10. 2021ROAD: The ROad event Awareness Dataset for Autonomous Drivingcited by 54
  11. 2021Semi-Supervised Action Recognition with Temporal Contrastive Learningcited by 50
  12. 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
  13. 2021AGENT: A Benchmark for Core Psychological Reasoningcited by 42
  14. 2023Egocentric Audio-Visual Object Localizationcited by 40
  15. 2021The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methodscited by 37
  16. 2024EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarioscited by 37
  17. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  18. 2024EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understandingcited by 35
  19. 2023EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understandingcited by 32
  20. 2024Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videoscited by 32
  21. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  22. 2023Ego-Only: Egocentric Action Detection without Exocentric Transferringcited by 30
  23. 2024VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AIcited by 30
  24. 2023A Review of Deep Learning for Video Captioningcited by 28
  25. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 26
  26. 2024ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectivescited by 26
  27. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  28. 2022EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queriescited by 25
  29. 2025Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understandingcited by 25
  30. 2022Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localizationcited by 24
  31. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24
  32. 2025FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understandingcited by 24
  33. 2022EgoEnv: Human-centric environment representations from egocentric videocited by 22
  34. 2023Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipationcited by 20
  35. 2023Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognitioncited by 20
  36. 2025EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World?cited by 20
  37. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  38. 2025HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generationcited by 19

50 most cited of 127 All 127 (JSON)