# Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization

arXiv: https://arxiv.org/abs/1908.06387 · HTML: https://mlsanity.com/paper/1908.06387/

## Key facts

- Cited by 24 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 18 Aug 2019.

## Built on by (most cited first, 24 of 24)

- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [A Spacecraft Dataset for Detection, Segmentation and Parts Recognition](https://mlsanity.com/paper/2106.08186/) (2021)
- [A Hierarchical Dual Model of Environment- and Place-Specific Utility for Visual Place Recognition](https://mlsanity.com/paper/2107.02440/) (2021)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation](https://mlsanity.com/paper/2311.10696/) (2023)
- [Learning Semantics for Visual Place Recognition through Multi-Scale Attention](https://mlsanity.com/paper/2201.09701/) (2022)
- [Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifiers](https://mlsanity.com/paper/2305.13849/) (2023)
- [DoubleMatch: Improving Semi-Supervised Learning with Self-Supervision](https://mlsanity.com/paper/2205.05575/) (2022)
- [Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors](https://mlsanity.com/paper/2209.13586/) (2022)
- [MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics](https://mlsanity.com/paper/2407.15663/) (2024)
- [MixSiam: A Mixture-based Approach to Self-supervised Representation Learning](https://mlsanity.com/paper/2111.02679/) (2021)
- [Self-supervised Learning for Panoptic Segmentation of Multiple Fruit Flower Species](https://mlsanity.com/paper/2209.04618/) (2022)
- [Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View](https://mlsanity.com/paper/2408.14789/) (2024)
- [Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency](https://mlsanity.com/paper/2402.09237/) (2024)
- [SimFLE: Simple Facial Landmark Encoding for Self-Supervised Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2303.07648/) (2023)
- [FindView: Precise Target View Localization Task for Look Around Agents](https://mlsanity.com/paper/2303.09054/) (2023)
- [Locking On: Leveraging Dynamic Vehicle-Imposed Motion Constraints to Improve Visual Localization](https://mlsanity.com/paper/2306.17529/) (2023)
- [PRAM: Place Recognition Anywhere Model for Efficient Visual Localization](https://mlsanity.com/paper/2404.07785/) (2024)
- [BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignment](https://mlsanity.com/paper/2410.20969/) (2024)
- [OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environmental Inference](https://mlsanity.com/paper/2507.02929/) (2025)
- [Towards Robust Artificial Intelligence: Self-Supervised Learning Approach for Out-of-Distribution Detection](https://mlsanity.com/paper/2510.12713/) (2025)

## Abstract (arXiv)

Long-term visual localization is the problem of estimating the camera pose of a given query image in a scene whose appearance changes over time. It is an important problem in practice, for example, encountered in autonomous driving. In order to gain robustness to such changes, long-term localization approaches often use segmantic segmentations as an invariant scene representation, as the semantic meaning of each scene part should not be affected by seasonal and other changes. However, these representations are typically not very discriminative due to the limited number of available classes. In this paper, we propose a new neural network, the Fine-Grained Segmentation Network (FGSN), that can be used to provide image segmentations with a larger number of labels and can be trained in a self-supervised fashion. In addition, we show how FGSNs can be trained to output consistent labels across seasonal changes. We demonstrate through extensive experiments that integrating the fine-grained segmentations produced by our FGSNs into existing localization algorithms leads to substantial improvements in localization performance.