# LOSS-SLAM: Lightweight Open-Set Semantic Simultaneous Localization and Mapping

arXiv: https://arxiv.org/abs/2404.04377 · HTML: https://mlsanity.com/paper/2404.04377/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding](https://mlsanity.com/paper/2210.03043/) (2022)
- [Probabilistic Data Association via Mixture Models for Robust Semantic SLAM](https://mlsanity.com/paper/1909.11213/) (2019)
- [An Object SLAM Framework for Association, Mapping, and High-Level Tasks](https://mlsanity.com/paper/2305.07299/) (2023)
- [NeuSE: Neural SE(3)-Equivariant Embedding for Long-Term Object-Based Simultaneous Localization and Mapping](https://mlsanity.com/paper/2303.07308/) (2023)
- [Neural Implicit Vision-Language Feature Fields](https://mlsanity.com/paper/2303.10962/) (2023)
- [SCIM: Simultaneous Clustering, Inference, and Mapping for Open-World Semantic Scene Understanding](https://mlsanity.com/paper/2206.10670/) (2022)
- [Multi-Robot Distributed Semantic Mapping in Unfamiliar Environments through Online Matching of Learned Representations](https://mlsanity.com/paper/2103.14805/) (2021)
- [Discrete-Continuous Smoothing and Mapping](https://mlsanity.com/paper/2204.11936/) (2022)

## Built on by (most cited first, 2 of 2)

- [DINO-SLAM: DINO-informed RGB-D SLAM for Neural Implicit and Explicit Representations](https://mlsanity.com/paper/2507.19474/) (2025)
- [Open-Set Semantic Uncertainty Aware Metric-Semantic Graph Matching](https://mlsanity.com/paper/2409.11555/) (2024)

## Abstract (arXiv)

Enabling robots to understand the world in terms of objects is a critical building block towards higher level autonomy. The success of foundation models in vision has created the ability to segment and identify nearly all objects in the world. However, utilizing such objects to localize the robot and build an open-set semantic map of the world remains an open research question. In this work, a system of identifying, localizing, and encoding objects is tightly coupled with probabilistic graphical models for performing open-set semantic simultaneous localization and mapping (SLAM). Results are presented demonstrating that the proposed lightweight object encoding can be used to perform more accurate object-based SLAM than existing open-set methods, closed-set methods, and geometric methods while incurring a lower computational overhead than existing open-set mapping methods.