DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping
Not on the map. Why?
Abstract (arXiv)
Open-set semantic mapping enables language-driven robotic perception, but current instance-centric approaches are bottlenecked by context-depriving and computationally expensive crop-based feature extraction. To overcome this fundamental limitation, we introduce DISC (Dense Integrated Semantic Context), featuring a novel single-pass, distance-weighted extraction mechanism. By deriving high-fidelity CLIP embeddings directly from the vision transformer's intermediate layers, our approach eliminates the latency and domain-shift artifacts of traditional image cropping, yielding pure, mask-aligned semantic representations. To fully leverage these features in large-scale continuous mapping, DISC is built upon a fully GPU-accelerated architecture that replaces periodic offline processing with precise, on-the-fly voxel-level instance refinement. We evaluate our approach on standard benchmarks (Replica, ScanNet) and a newly generated large-scale-mapping dataset based on Habitat-Matterport 3D (HM3DSEM) to assess scalability across complex scenes in multi-story buildings. Extensive evaluations demonstrate that DISC significantly surpasses current state-of-the-art zero-shot methods in both semantic accuracy and query retrieval, providing a robust, real-time capable framework for robotic deployment. The full source code, data generation and evaluation pipelines will be made available at https://github.com/DFKI-NI/DISC.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
32 of 36 references.
References · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2023Segment Everything Everywhere All at Oncecited by 548
- 2023LERF: Language Embedded Radiance Fieldscited by 543
20 more
- 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
- 2022Visual Language Maps for Robot Navigationcited by 365
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
- 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 352
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
- 2023Fast Segment Anythingcited by 290
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
- 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
- 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
- 2022Habitat-Matterport 3D Semantics Datasetcited by 110
- 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
- 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
- 2023Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representationcited by 54
- 2024Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environmentscited by 36
- 2024Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graphcited by 32
- 2024The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphscited by 11
- 2024Online Knowledge Integration for 3D Semantic Mapping: A Surveycited by 2
- 2025CORE-3D: Context-aware Open-vocabulary Retrieval by Embeddings in 3Dcited by 2
- 2022Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Raytracing for Embedded Hardware on Mobile Robotscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Instant Colorization of Gaussian Splatscited by 0
- 2026Instant Colorization of Gaussian Splats18 Apr 2026