mlsanityData to 14 Oct 2026

Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action

arXiv 2606.15647 ↗First posted 5 Apr 2026cs.AI cs.CV cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Foundation models have demonstrated impressive performance in enhancing healthcare efficiency across a wide range of medical applications. Nevertheless, their limited ability to perceive, understand, and interact with the physical world significantly constrains their effectiveness in real-world clinical workflows, where safety-critical decision-making and physical execution are tightly coupled. Recently, embodied artificial intelligence (AI) has emerged as a promising physical-interactive paradigm for intelligent healthcare, enabling agents to operate in complex medical environments. As research in this area rapidly expands, understanding how intelligent agents function as integrated, end-to-end systems in clinical environments becomes increasingly critical. However, existing surveys on medical embodied AI largely emphasize individual aspects or functional components, lacking a unified system-level organization of the field. To support and consolidate recent advances, we systematically survey the core components of medical embodied AI, with a particular emphasis on the coordinated integration of perception, decision-making, and action. We further review representative medical applications and relevant datasets, and we analyze the major challenges encountered in real-world clinical practice. Finally, we discuss key directions for future research in this rapidly evolving field. The associated project can be found at https://github.com/VMVLab/Medical_Embodied_AI_Paper_List.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 88 of its 218 references that are in the map. First posted on arXiv 5 Apr 2026.

Builds on

88 of 218 references.

References · 88

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  3. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  4. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  5. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  6. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
  7. 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 352
  8. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  9. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  10. 2018Deep Facial Expression Recognition: A Surveycited by 270
  11. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  12. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 173
38 more
  1. 20192017 Robotic Instrument Segmentation Challengecited by 168
  2. 2020TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networkscited by 141
  3. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 121
  4. 2020VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Imagescited by 111
  5. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 110
  6. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  7. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 94
  8. 2021Medical Visual Question Answering: A Surveycited by 93
  9. 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
  10. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  11. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  12. 2020FocalMix: Semi-Supervised Learning for 3D Medical Image Detectioncited by 52
  13. 2018MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimationcited by 49
  14. 20224D-OR: Semantic Scene Graphs for OR Domain Modelingcited by 49
  15. 2023SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgerycited by 48
  16. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  17. 2021CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomographycited by 46
  18. 2021Deep Learning for Micro-expression Recognition: A Surveycited by 43
  19. 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
  20. 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
  21. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 40
  22. 2016The TUM LapChole dataset for the M2CAI 2016 workflow challengecited by 38
  23. 2024Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splattingcited by 38
  24. 2024GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answeringcited by 37
  25. 2024LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learningcited by 34
  26. 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 32
  27. 2021SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learningcited by 31
  28. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 31
  29. 2023Lumbar spine segmentation in MR images: a dataset and a public benchmarkcited by 30
  30. 2023DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversationscited by 30
  31. 2024RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Modelcited by 29
  32. 2021MIcro-Surgical Anastomose Workflow recognition challenge reportcited by 28
  33. 2025MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environmentscited by 28
  34. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  35. 2025ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoningcited by 27
  36. 2020m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networkscited by 26
  37. 2022OSSO: Obtaining Skeletal Shape from Outsidecited by 25
  38. 2024General surgery vision transformer: A video pre-trained foundation model for general surgerycited by 25

50 most cited of 88 All 88 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.