The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models
Not on the map. Why?
Abstract (arXiv)
Semantic information in embodied AI is inherently multi-source and multi-stage, making it challenging to fully leverage for achieving stable perception-to-action loops in real-world environments. Early studies have combined manual engineering with deep neural networks, achieving notable progress in specific semantic-related embodied tasks. However, as embodied agents encounter increasingly complex environments and open-ended tasks, the demand for more generalizable and robust semantic processing capabilities has become imperative. Recent advances in foundation models (FMs) address this challenge through their cross-domain generalization abilities and rich semantic priors, reshaping the landscape of embodied AI research. In this survey, we propose the Semantic Lifecycle as a unified framework to characterize the evolution of semantic knowledge within embodied AI driven by foundation models. Departing from traditional paradigms that treat semantic processing as isolated modules or disjoint tasks, our framework offers a holistic perspective that captures the continuous flow and maintenance of semantic knowledge. Guided by this embodied semantic lifecycle, we further analyze and compare recent advances across three key stages: acquisition, representation, and storage. Finally, we summarize existing challenges and outline promising directions for future research.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 160 of its 194 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
160 of 194 references.
References · 160
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2023Segment Anythingcited by 8,234
- 2023Visual Instruction Tuningcited by 7,869
- 2017Mask R-CNNcited by 7,277
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
38 more
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2020Image Segmentation Using Deep Learning: A Surveycited by 660
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
- 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
- 2023LERF: Language Embedded Radiance Fieldscited by 543
- 2020Rescaling Egocentric Visioncited by 540
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
- 2023LangSplat: 3D Language Gaussian Splattingcited by 415
- 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
- 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
- 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
- 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
- 2024Navigation World Modelscited by 272
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
- 2020Experience Grounds Languagecited by 242
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
- 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 208
- 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
- 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
- 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
- 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
- 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
- 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 129
- 2023Transformer-Based Visual Segmentation: A Surveycited by 124
- 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
- 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
- 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
- 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 95
- 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
- 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
- 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
- 2024Multimodal Alignment and Fusion: A Surveycited by 74
50 most cited of 160 All 160 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.