mlsanityData to 14 Oct 2026

The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models

arXiv 2601.08876 ↗First posted 12 Jan 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Semantic information in embodied AI is inherently multi-source and multi-stage, making it challenging to fully leverage for achieving stable perception-to-action loops in real-world environments. Early studies have combined manual engineering with deep neural networks, achieving notable progress in specific semantic-related embodied tasks. However, as embodied agents encounter increasingly complex environments and open-ended tasks, the demand for more generalizable and robust semantic processing capabilities has become imperative. Recent advances in foundation models (FMs) address this challenge through their cross-domain generalization abilities and rich semantic priors, reshaping the landscape of embodied AI research. In this survey, we propose the Semantic Lifecycle as a unified framework to characterize the evolution of semantic knowledge within embodied AI driven by foundation models. Departing from traditional paradigms that treat semantic processing as isolated modules or disjoint tasks, our framework offers a holistic perspective that captures the continuous flow and maintenance of semantic knowledge. Guided by this embodied semantic lifecycle, we further analyze and compare recent advances across three key stages: acquisition, representation, and storage. Finally, we summarize existing challenges and outline promising directions for future research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 160 of its 194 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

160 of 194 references.

References · 160

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2023Segment Anythingcited by 8,234
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2017Mask R-CNNcited by 7,277
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
38 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  3. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  6. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  7. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  8. 2023LERF: Language Embedded Radiance Fieldscited by 543
  9. 2020Rescaling Egocentric Visioncited by 540
  10. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  11. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  12. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  13. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
  14. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  15. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  16. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  17. 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
  18. 2024Navigation World Modelscited by 272
  19. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  20. 2020Experience Grounds Languagecited by 242
  21. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
  22. 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 208
  23. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  24. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
  25. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  26. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  27. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  28. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 129
  29. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  30. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
  31. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  32. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  33. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  34. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 95
  35. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  36. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
  37. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
  38. 2024Multimodal Alignment and Fusion: A Surveycited by 74

50 most cited of 160 All 160 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.