Multimodal Datasets and Benchmarks for Reasoning about Dynamic Spatio-Temporality in Everyday Environments
Not on the map. Why?
Abstract (arXiv)
We used a 3D simulator to create artificial video data with standardized annotations, aiming to aid in the development of Embodied AI. Our question answering (QA) dataset measures the extent to which a robot can understand human behavior and the environment in a home setting. Preliminary experiments suggest our dataset is useful in measuring AI's comprehension of daily life. endabstract
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 21 Aug 2024.
Builds on
12 of 18 references.
References · 12
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
- 2018VirtualHome: Simulating Household Activities via Programscited by 411
- 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
- 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
- 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
- 2022Retrospectives on the Embodied AI Workshopcited by 46
- 2024A call for embodied AIcited by 7
- 2023Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Spacecited by 5
- 2024Synthetic Multimodal Dataset for Empowering Safety and Well-being in Home Environmentscited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.