mlsanityData to 14 Oct 2026

Multimodal Data Storage and Retrieval for Embodied AI: A Survey

arXiv 2508.13901 ↗First posted 19 Aug 2025cs.RO cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Embodied AI (EAI) agents continuously interact with the physical world, generating vast, heterogeneous multimodal data streams that traditional management systems are ill-equipped to handle. In this survey, we first systematically evaluate five storage architectures (Graph Databases, Multi-Model Databases, Data Lakes, Vector Databases, and Time-Series Databases), focusing on their suitability for addressing EAI's core requirements, including physical grounding, low-latency access, and dynamic scalability. We then analyze five retrieval paradigms (Fusion Strategy-Based Retrieval, Representation Alignment-Based Retrieval, Graph-Structure-Based Retrieval, Generation Model-Based Retrieval, and Efficient Retrieval-Based Optimization), revealing a fundamental tension between achieving long-term semantic coherence and maintaining real-time responsiveness. Based on this comprehensive analysis, we identify key bottlenecks, spanning from the foundational Physical Grounding Gap to systemic challenges in cross-modal integration, dynamic adaptation, and open-world generalization. Finally, we outline a forward-looking research agenda encompassing physics-aware data models, adaptive storage-retrieval co-optimization, and standardized benchmarking, to guide future research toward principled data management solutions for EAI. Our survey is based on a comprehensive review of more than 180 related studies, providing a rigorous roadmap for designing the robust, high-performance data management frameworks essential for the next generation of autonomous embodied systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 78 of its 184 references that are in the map. First posted on arXiv 19 Aug 2025.

Builds on

78 of 184 references.

References · 78

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2017Inductive Representation Learning on Large Graphscited by 6,320
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
38 more
  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  10. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  11. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  12. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  13. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  14. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  15. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  16. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  17. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  18. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  19. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  20. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  21. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  22. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 328
  23. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  24. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  25. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  26. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  27. 2017A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applicationscited by 260
  28. 2022Transformer Memory as a Differentiable Search Indexcited by 247
  29. 2024RT-H: Action Hierarchies Using Languagecited by 210
  30. 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
  31. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  32. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  33. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  34. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
  35. 2022A Neural Corpus Indexer for Document Retrievalcited by 111
  36. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  37. 2024Multimodal Alignment and Fusion: A Surveycited by 74
  38. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 63

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.