mlsanityData to 14 Oct 2026

The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding

arXiv 2503.17116 ↗First posted 21 Mar 2025cs.MM cs.AI cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Egocentric video has seen increased interest in recent years, as it is used in a range of areas. However, most existing datasets are limited to a single perspective. In this paper, we present the CASTLE 2024 dataset, a multimodal collection containing ego- and exo-centric (i.e., first- and third-person perspective) video and audio from 15 time-aligned sources, as well as other sensor streams and auxiliary data. The dataset was recorded by volunteer participants over four days in a fixed location and includes the point of view of 10 participants, with an additional 5 fixed cameras providing an exocentric perspective. The entire dataset contains over 600 hours of UHD video recorded at 50 frames per second. In contrast to other datasets, CASTLE 2024 does not contain any partial censoring, such as blurred faces or distorted audio. The dataset is available via https://castle-dataset.github.io/.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

6 of 14 references.

References · 6

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  3. 2020Rescaling Egocentric Visioncited by 540
  4. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  5. 2020MEVA: A Large-Scale Multiview, Multimodal Video Dataset for Activity Detectioncited by 44
  6. 2025EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activitycited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 15.

16 papers.

Sort:

By area:

  1. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 7
  2. 2025FIGROTD: A Friendly-to-Handle Dataset for Image Guided Retrieval with Optional Textcited by 1
  3. 2026MARS: Technical Report for the CASTLE Challenge at EgoVis 2026cited by 1
  4. 2026CuriosAI Submission to the CASTLE Challenge at EgoVis 2026cited by 1
  5. 2026CASTLE2026 Team WDL Technical Reportcited by 1
  6. 20263rd Place at CVPR 2026 CASTLE Challenge: Agentic Multi-View Long-Context Video Understanding via Hierarchical Knowledge Graph Retrievalcited by 1
  7. 2026Logi-PAR: Logic-Infused Patient Activity Recognition via Differentiable Rulecited by 0
  8. 2026Bringing a Personal Point of View: Evaluating Dynamic 3D Gaussian Splatting for Egocentric Scene Reconstructioncited by 0
  9. 2026Imprint: Online Memory Compression for Long-Horizon Egocentric QAcited by 0
  10. 2026OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMscited by 0
  11. 2026Results of the 1st Asynchronous CASTLE Challenge at the Joint Egocentric Vision Workshop in Conjunction with CVPR 2026cited by 0
  12. 2026Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulationscited by 0
4 more
  1. 2026Talking to Me or Someone Else? Rethinking Talk-to-Me Detection in Egocentric Videoscited by 0
  2. 2026CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Videocited by 0
  3. 2026NextMe-800: Anticipating Personal Behavior from Months of Egocentric Videocited by 0
  4. 2026GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?cited by 0