# Multiview Scene Graph arXiv: https://arxiv.org/abs/2410.11187 ยท HTML: https://mlsanity.com/paper/2410.11187/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 58 of its 74 references that are in the map. - First posted on arXiv 15 Oct 2024. ## Builds on (most cited first, 30 of 58) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023) - [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019) - [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019) - [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016) - [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021) - [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015) - [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015) - [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016) - [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021) - [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017) - [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021) - [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022) - [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019) - [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017) - [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018) - [Tracking Anything with Decoupled Video Segmentation](https://mlsanity.com/paper/2309.03903/) (2023) - [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020) ## Built on by (most cited first, 9 of 9) - [Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMs](https://mlsanity.com/paper/2505.21955/) (2025) - [Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI](https://mlsanity.com/paper/2511.20620/) (2025) - [Co-VisiON: Co-Visibility ReasONing on Sparse Image Sets of Indoor Scenes](https://mlsanity.com/paper/2506.16805/) (2025) - [MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation](https://mlsanity.com/paper/2602.07082/) (2026) - [CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations](https://mlsanity.com/paper/2506.21357/) (2025) - [M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments](https://mlsanity.com/paper/2508.17044/) (2025) - [RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflection](https://mlsanity.com/paper/2602.19974/) (2026) - [PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching](https://mlsanity.com/paper/2603.20818/) (2026) - [HSG: Hyperbolic Scene Graph](https://mlsanity.com/paper/2604.17454/) (2026) ## Abstract (arXiv) A proper scene representation is central to the pursuit of spatial intelligence where agents can robustly reconstruct and efficiently understand 3D scenes. A scene representation is either metric, such as landmark maps in 3D reconstruction, 3D bounding boxes in object detection, or voxel grids in occupancy prediction, or topological, such as pose graphs with loop closures in SLAM or visibility graphs in SfM. In this work, we propose to build Multiview Scene Graphs (MSG) from unposed images, representing a scene topologically with interconnected place and object nodes. The task of building MSG is challenging for existing representation learning methods since it needs to jointly address both visual place recognition, object detection, and object association from images with limited fields of view and potentially large viewpoint changes. To evaluate any method tackling this task, we developed an MSG dataset and annotation based on a public 3D dataset. We also propose an evaluation metric based on the intersection-over-union score of MSG edges. Moreover, we develop a novel baseline method built on mainstream pretrained vision models, combining visual place recognition and object association into one Transformer decoder architecture. Experiments demonstrate that our method has superior performance compared to existing relevant baselines.