KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
We push the frontier of large-scale spatial intelligence in Vision-Language Models (VLMs) and introduce the first benchmark that probes geographical layout understanding from real-world videos, spanning up to 1km distances. Inspired by the cognitive science literature, we evaluate models against the hierarchical stages of human spatial awareness: anchoring via landmarks, connecting them through routes, and integrating these into global mental maps. Extensive experiments reveal a fundamental divergence in how current AI models process spatial information. Instead of utilising true path integration or forming geometric survey knowledge, we find that VLMs rely almost entirely on 2D visual recognition and text-matching to bypass complex spatial reasoning. The benchmark is publicly available at https://perception-test-challenge.github.io/kilometervision.html.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 66 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
40 of 66 references.
Closest ancestors · 3
Same direction, newest first
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 2024FLAME: Learning to Navigate with Multimodal LLM in Urban Environments20 Aug
- 2023Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs24 Nov
Other references · 37
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2023VideoChat: Chat-Centric Video Understandingcited by 857
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
25 more
- 2020Rescaling Egocentric Visioncited by 540
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
- 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
- 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
- 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 314
- 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
- 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
- 2019Cross-task weakly supervised learning from instructional videoscited by 199
- 2017A Survey of Structure from Motioncited by 158
- 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
- 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
- 2024Evaluating the World Model Implicit in a Generative Modelcited by 113
- 2025PerceptionLM: Open-Access Data and Models for Detailed Visual Understandingcited by 92
- 2023GPT4GEO: How a Language Model Sees the World's Geographycited by 68
- 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
- 2024Does Spatial Cognition Emerge in Frontier Models?cited by 66
- 2023Large Language Models are Temporal and Causal Reasoners for Video Question Answeringcited by 51
- 2023LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videoscited by 51
- 2017From Lifestyle Vlogs to Everyday Interactionscited by 42
- 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
- 2023Vamos: Versatile Action Models for Video Understandingcited by 40
- 2025Benchmarking Egocentric Visual-Inertial SLAM at City Scalecited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
- 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
- 2026Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
- 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods