Multimodal Data Storage and Retrieval for Embodied AI: A Survey
Not on the map. Why?
Abstract (arXiv)
Embodied AI (EAI) agents continuously interact with the physical world, generating vast, heterogeneous multimodal data streams that traditional management systems are ill-equipped to handle. In this survey, we first systematically evaluate five storage architectures (Graph Databases, Multi-Model Databases, Data Lakes, Vector Databases, and Time-Series Databases), focusing on their suitability for addressing EAI's core requirements, including physical grounding, low-latency access, and dynamic scalability. We then analyze five retrieval paradigms (Fusion Strategy-Based Retrieval, Representation Alignment-Based Retrieval, Graph-Structure-Based Retrieval, Generation Model-Based Retrieval, and Efficient Retrieval-Based Optimization), revealing a fundamental tension between achieving long-term semantic coherence and maintaining real-time responsiveness. Based on this comprehensive analysis, we identify key bottlenecks, spanning from the foundational Physical Grounding Gap to systemic challenges in cross-modal integration, dynamic adaptation, and open-world generalization. Finally, we outline a forward-looking research agenda encompassing physics-aware data models, adaptive storage-retrieval co-optimization, and standardized benchmarking, to guide future research toward principled data management solutions for EAI. Our survey is based on a comprehensive review of more than 180 related studies, providing a rigorous roadmap for designing the robust, high-performance data management frameworks essential for the next generation of autonomous embodied systems.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 78 of its 184 references that are in the map. First posted on arXiv 19 Aug 2025.
Builds on
78 of 184 references.
References · 78
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
38 more
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
- 2014DeepWalk: Online Learning of Social Representationscited by 1,547
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
- 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
- 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
- 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
- 2023Large Language Models for Information Retrieval: A Surveycited by 331
- 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 328
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
- 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
- 2017A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applicationscited by 260
- 2022Transformer Memory as a Differentiable Search Indexcited by 247
- 2024RT-H: Action Hierarchies Using Languagecited by 210
- 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
- 2021Embodied Intelligence via Learning and Evolutioncited by 163
- 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
- 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
- 2022A Neural Corpus Indexer for Document Retrievalcited by 111
- 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
- 2024Multimodal Alignment and Fusion: A Surveycited by 74
- 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 63
50 most cited of 78 All 78 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.