Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence
Not on the map. Why?
Abstract (arXiv)
Autonomous UAV systems increasingly rely on multimodal large language models (MLLMs) to operate in complex real-world environments. Such embodied scenarios require not only understanding the surrounding space but also maintaining a coherent representation of the agent itself. However, existing UAV-oriented approaches and benchmarks remain largely environment-centric, primarily focusing on spatial understanding tasks, with the agent's self-awareness remaining implicit. To address this gap, we introduce SIS-Bench, a benchmark for evaluating embodied spatial intelligence in UAV scenarios under a unified self-in-space formulation. SIS-Bench organizes evaluation along two complementary dimensions, space and self, and a three-level hierarchy of perception, memory, and reasoning. It contains 4,856 question--answer pairs across 13 tasks derived from 1,646 real-world UAV videos through a task-conditioned construction pipeline with expert verification. Extensive evaluations reveal that current MLLMs exhibit fundamental limitations in modeling dynamic and agent-centered processes. In particular, we observe a clear imbalance between spatial cognition and self-awareness, as well as a progressive performance degradation across cognitive levels. Motivated by these findings, we further explore a motion-aware representation that incorporates self-related dynamics through optical flow and visual feature fusion. Experimental results show that modeling agent motion consistently improves perception and memory performance, not only in spatial cognition but also in self-awareness, and generalizes to downstream UAV decision-making tasks. Our results highlight the importance of self-awareness for advancing embodied spatial intelligence, and provide both a new benchmark and empirical evidence for motion-aware self-in-space modeling.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
49 of 60 references.
References · 49
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
37 more
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2025Kimi-VL Technical Reportcited by 388
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
- 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
- 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
- 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
- 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 144
- 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
- 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
- 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 125
- 2024Grounded 3D-LLM with Referent Tokenscited by 115
- 2025MiMo-VL Technical Reportcited by 115
- 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
- 2025Ovis2.5 Technical Reportcited by 108
- 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
- 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
- 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
- 2021Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigationcited by 47
- 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 46
- 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
- 2026STEP3-VL-10B Technical Reportcited by 32
- 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecturecited by 30
- 2024ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectivescited by 29
- 2025UAVScenes: A Multi-Modal Dataset for UAVscited by 27
- 2025Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesiscited by 15
- 2025V2-SAM: Marrying SAM2 with Multi-Prompt Experts for Cross-View Object Correspondencecited by 14
- 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVscited by 11
- 2025IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environmentscited by 10
- 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?cited by 10
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
- 2025AirScape: An Aerial Generative World Model with Motion Controllabilitycited by 7
- 2025Agentic UAVs: LLM-Driven Autonomy with Integrated Tool-Calling and Cognitive Reasoningcited by 7
- 2026Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorialcited by 4
- 2025Embodiment in multimodal large language modelscited by 1
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understandingcited by 5
- 2026Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMscited by 2
- 2026RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?cited by 1
- 2026The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answeringcited by 0
- 2026EgoSIS: From Factorized Visual Ego-Transitions to Motion-Canonical Spatial Evidence for UAV Reasoningcited by 0
- 2026UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agentscited by 0
- 2026UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agents16 Sep 2026
- 2026EgoSIS: From Factorized Visual Ego-Transitions to Motion-Canonical Spatial Evidence for UAV Reasoning8 Sep 2026
- 2026The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering5 Aug 2026
- 2026RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?3 Aug 2026
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding30 Jul 2026
- 2026Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs17 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models
- 2026AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration
- 2026Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
- 2026ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception
- 2025NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
- 2025BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs
- 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation
- 2026Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents