mlsanityData to 7 Oct 2026

CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence

arXiv 2606.10401 ↗First posted 9 Jun 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Spatial intelligence is a key frontier for multimodal large language models (MLLMs), enabling them to reason about the physical world from visual experience. Inspired by human spatial cognition, recent approaches construct grid-based cognitive maps from multi-frame visual inputs to maintain coherent spatial representations over time. However, limited context lengths still challenge spatial understanding, while existing methods, such as long-context modeling and external memory, often require architectural changes, memory modules, or finetuning, limiting their applicability to off-the-shelf pretrained MLLMs. This motivates a lightweight, model-agnostic method for preserving spatial information beyond the native context window. To this end, we propose a plug-and-play multi-agent framework that collaboratively constructs cognitive maps as structured spatial memory, enhancing the spatial understanding of arbitrary pretrained MLLMs without architectural modification or additional training. Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification. Extensive experiments demonstrate that our method achieves superior performance on spatial understanding tasks while remaining fully training-free. Code will be released.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 32 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

30 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 27

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  9. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  10. 2025OpenAI GPT-5 System Cardcited by 910
  11. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  12. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
15 more
  1. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  2. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  3. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
  4. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  5. 2020Learning to Explore using Active Neural SLAMcited by 308
  6. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  7. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  8. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
  9. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  10. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 88
  11. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
  12. 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
  13. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
  14. 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 34
  15. 2025LongVideoAgent: Multi-Agent Reasoning with Long Videoscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  2. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
  3. 2026SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
  4. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science
  5. 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning
  6. 2026Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models
  7. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
  8. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes