mlsanityData to 14 Oct 2026

CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

arXiv 2607.06534 ↗First posted 7 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a topology-aware 3D-LLM for multi-room 3D scene understanding. CAIRN aligns transformer attention with scene hierarchy, giving the model explicit awareness of object-level relations and room-level connectivity. It enriches object tokens with room-local relational context via a graph neural network, introduces learned room tokens for room-level abstraction, and applies a hierarchical attention mask with geometric bias to route information according to scene topology. CAIRN is developed on CAIRN-MR, a benchmark we introduce on HM3D for multi-room 3D scene understanding, covering grounding, captioning, and four question-answering tasks that progressively evaluate from intra-room perception to cross-room reasoning. Experiments show that CAIRN outperforms prior 3D-LLMs by a large margin across all CAIRN-MR tasks while remaining competitive on five single-room benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 66 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

59 of 66 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Pts3D-LLM: Studying the Impact of Token Structure for 3D Scene Understanding With Large Language Models6 Jun
  2. 2025Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning1 Mar
  3. 20253UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding14 Jan
  4. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
  5. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  6. 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences2 Dec
  7. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors18 Mar
  8. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 51

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2020Longformer: The Long-Document Transformercited by 3,413
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  10. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  11. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  12. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
38 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  2. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  3. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  4. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  5. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  6. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  7. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  8. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  9. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  10. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
  11. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
  12. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  13. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 218
  14. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  15. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  16. 2022Structure-Aware Transformer for Graph Representation Learningcited by 194
  17. 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
  18. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
  19. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  20. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  21. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  22. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
  23. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  24. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  25. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  26. 2023BuboGPT: Enabling Visual Grounding in Multi-Modal LLMscited by 114
  27. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 106
  28. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 82
  29. 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 70
  30. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  31. 2023VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloudcited by 43
  32. 2023Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioningcited by 39
  33. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 27
  34. 2024Robin3D: Improving 3D Large Language Model via Robust Instruction Tuningcited by 26
  35. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 14
  36. 2024Gradformer: Graph Transformer with Exponential Decaycited by 11
  37. 2024CAMON: Cooperative Agents for Multi-Object Navigation with LLM-based Conversationscited by 7
  38. 2025COOPERA: Continual Open-Ended Human-Robot Assistancecited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SpatialReasoner: Active Perception for Large-Scale 3D Scene Understanding
  2. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
  3. 2025SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenes
  4. 2026RieMind: Geometry-Grounded Spatial Agent for Scene Understanding
  5. 20243DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding
  6. 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
  7. 2025Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering
  8. 2026Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding