# MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition

arXiv: https://arxiv.org/abs/2104.05327 · HTML: https://mlsanity.com/paper/2104.05327/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval](https://mlsanity.com/paper/1904.09793/) (2019)
- [SOE-Net: A Self-Attention and Orientation Encoding Network for Point Cloud based Place Recognition](https://mlsanity.com/paper/2011.12430/) (2020)
- [Aggregating Deep Convolutional Features for Image Retrieval](https://mlsanity.com/paper/1510.07493/) (2015)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [SegMatch: Segment based loop-closure for 3D point clouds](https://mlsanity.com/paper/1609.07720/) (2016)
- [DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization](https://mlsanity.com/paper/2007.09217/) (2020)
- [Augmenting Visual Place Recognition with Structural Cues](https://mlsanity.com/paper/2003.00278/) (2020)
- [Efficient 3D Point Cloud Feature Learning for Large-Scale Place Recognition](https://mlsanity.com/paper/2101.02374/) (2021)
- [CORAL: Colored structural representation for bi-modal place recognition](https://mlsanity.com/paper/2011.10934/) (2020)
- [PIC-Net: Point Cloud and Image Collaboration Network for Large-Scale Place Recognition](https://mlsanity.com/paper/2008.00658/) (2020)
- [Visual Place Recognition with Probabilistic Vertex Voting](https://mlsanity.com/paper/1610.03548/) (2016)

## Built on by (most cited first, 30 of 41)

- [AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition](https://mlsanity.com/paper/2111.11739/) (2021)
- [TransLoc3D : Point Cloud based Large-scale Place Recognition using Adaptive Receptive Fields](https://mlsanity.com/paper/2105.11605/) (2021)
- [RING++: Roto-translation Invariant Gram for Global Localization on a Sparse Scan Map](https://mlsanity.com/paper/2210.05984/) (2022)
- [UAVScenes: A Multi-Modal Dataset for UAVs](https://mlsanity.com/paper/2507.22412/) (2025)
- [General Place Recognition Survey: Towards Real-World Autonomy](https://mlsanity.com/paper/2405.04812/) (2024)
- [LCPR: A Multi-Scale Attention-Based LiDAR-Camera Fusion Network for Place Recognition](https://mlsanity.com/paper/2311.03198/) (2023)
- [MinkLoc3D-SI: 3D LiDAR place recognition with sparse convolutions, spherical coordinates, and intensity](https://mlsanity.com/paper/2112.06539/) (2021)
- [DistilVPR: Cross-Modal Knowledge Distillation for Visual Place Recognition](https://mlsanity.com/paper/2312.10616/) (2023)
- [CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Framework](https://mlsanity.com/paper/2503.02593/) (2025)
- [Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments](https://mlsanity.com/paper/2403.13395/) (2024)
- [MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics](https://mlsanity.com/paper/2407.15663/) (2024)
- [GSPR: Multimodal Place Recognition Using 3D Gaussian Splatting for Autonomous Driving](https://mlsanity.com/paper/2410.00299/) (2024)
- [MinkUNeXt: Point Cloud-based Large-scale Place Recognition using 3D Sparse Convolutions](https://mlsanity.com/paper/2403.07593/) (2024)
- [VXP: Voxel-Cross-Pixel Large-scale Image-LiDAR Place Recognition](https://mlsanity.com/paper/2403.14594/) (2024)
- [Leveraging BEV Representation for 360-degree Visual Place Recognition](https://mlsanity.com/paper/2305.13814/) (2023)
- [Explicit Interaction for Fusion-Based Place Recognition](https://mlsanity.com/paper/2402.17264/) (2024)
- [Ensemble-Based Event Camera Place Recognition Under Varying Illumination](https://mlsanity.com/paper/2509.01968/) (2025)
- [VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments](https://mlsanity.com/paper/2602.19735/) (2026)
- [Large-Scale Topological Radar Localization Using Learned Descriptors](https://mlsanity.com/paper/2110.03081/) (2021)
- [BEV2PR: BEV-Enhanced Visual Place Recognition with Structural Cues](https://mlsanity.com/paper/2403.06600/) (2024)
- [LRFusionPR: A Polar BEV-Based LiDAR-Radar Fusion Network for Place Recognition](https://mlsanity.com/paper/2504.19186/) (2025)
- [FusionLoc: Camera-2D LiDAR Fusion Using Multi-Head Self-Attention for End-to-End Serving Robot Relocalization](https://mlsanity.com/paper/2303.06872/) (2023)
- [PRISM-TopoMap: Online Topological Mapping with Place Recognition and Scan Matching](https://mlsanity.com/paper/2404.01674/) (2024)
- [PRFusion: Toward Effective and Robust Multi-Modal Place Recognition with Image and Point Cloud Fusion](https://mlsanity.com/paper/2410.04939/) (2024)
- [REGRACE: A Robust and Efficient Graph-based Re-localization Algorithm using Consistency Evaluation](https://mlsanity.com/paper/2503.03599/) (2025)
- [L2RSI: Cross-view LiDAR-based Place Recognition for Large-scale Urban Scenes via Remote Sensing Imagery](https://mlsanity.com/paper/2503.11245/) (2025)
- [Multi-modal Loop Closure Detection with Foundation Models in Severely Unstructured Environments](https://mlsanity.com/paper/2511.05404/) (2025)
- [UniMPR: A Unified Framework for Multimodal Place Recognition with Heterogeneous Sensor Configurations](https://mlsanity.com/paper/2512.18279/) (2025)
- [LaVPR: Benchmarking Language and Vision for Place Recognition](https://mlsanity.com/paper/2602.03253/) (2026)
- [VNI-Net: Vector Neurons-based Rotation-Invariant Descriptor for LiDAR Place Recognition](https://mlsanity.com/paper/2308.12870/) (2023)

## Abstract (arXiv)

We introduce a discriminative multimodal descriptor based on a pair of sensor readings: a point cloud from a LiDAR and an image from an RGB camera. Our descriptor, named MinkLoc++, can be used for place recognition, re-localization and loop closure purposes in robotics or autonomous vehicles applications. We use late fusion approach, where each modality is processed separately and fused in the final part of the processing pipeline. The proposed method achieves state-of-the-art performance on standard place recognition benchmarks. We also identify dominating modality problem when training a multimodal descriptor. The problem manifests itself when the network focuses on a modality with a larger overfit to the training data. This drives the loss down during the training but leads to suboptimal performance on the evaluation set. In this work we describe how to detect and mitigate such risk when using a deep metric learning approach to train a multimodal neural network. Our code is publicly available on the project website: https://github.com/jac99/MinkLocMultimodal.