# DistilVPR: Cross-Modal Knowledge Distillation for Visual Place Recognition

arXiv: https://arxiv.org/abs/2312.10616 · HTML: https://mlsanity.com/paper/2312.10616/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 23 of 23)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [Poincar'e GloVe: Hyperbolic Word Embeddings](https://mlsanity.com/paper/1810.06546/) (2018)
- [Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2206.02099/) (2022)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching](https://mlsanity.com/paper/2102.02973/) (2021)
- [Boreas: A Multi-Season Autonomous Driving Dataset](https://mlsanity.com/paper/2203.10168/) (2022)
- [MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition](https://mlsanity.com/paper/2104.05327/) (2021)
- [AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition](https://mlsanity.com/paper/2111.11739/) (2021)
- [HypLiLoc: Towards Effective LiDAR Pose Regression with Hyperbolic Fusion](https://mlsanity.com/paper/2304.00932/) (2023)
- [Augmenting Visual Place Recognition with Structural Cues](https://mlsanity.com/paper/2003.00278/) (2020)
- [Efficient 3D Point Cloud Feature Learning for Large-Scale Place Recognition](https://mlsanity.com/paper/2101.02374/) (2021)
- [CORAL: Colored structural representation for bi-modal place recognition](https://mlsanity.com/paper/2011.10934/) (2020)
- [LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation](https://mlsanity.com/paper/2304.11379/) (2023)
- [PIC-Net: Point Cloud and Image Collaboration Network for Large-Scale Place Recognition](https://mlsanity.com/paper/2008.00658/) (2020)
- [DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimization](https://mlsanity.com/paper/2212.06331/) (2022)
- [RobustMat: Neural Diffusion for Street Landmark Patch Matching under Challenging Environments](https://mlsanity.com/paper/2311.03904/) (2023)

## Built on by (most cited first, 8 of 8)

- [LightLoc: Learning Outdoor LiDAR Localization at Light Speed](https://mlsanity.com/paper/2503.17814/) (2025)
- [DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learning](https://mlsanity.com/paper/2408.07080/) (2024)
- [Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework](https://mlsanity.com/paper/2504.05187/) (2025)
- [NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2510.27647/) (2025)
- [SOAR: Regression-based LiDAR Relocalization for UAVs](https://mlsanity.com/paper/2602.13267/) (2026)
- [4DRaL: Bridging 4D Radar with LiDAR for Place Recognition using Knowledge Distillation](https://mlsanity.com/paper/2603.26206/) (2026)
- [LightLoc++: Sensor-Robust Representation Learning for Efficient Outdoor LiDAR Localization](https://mlsanity.com/paper/2608.15317/) (2026)
- [ReLoc: Rethinking Scene Coordinate Regression Architecture for Robust Outdoor LiDAR-based Localization](https://mlsanity.com/paper/2609.33344/) (2026)

## Abstract (arXiv)

The utilization of multi-modal sensor data in visual place recognition (VPR) has demonstrated enhanced performance compared to single-modal counterparts. Nonetheless, integrating additional sensors comes with elevated costs and may not be feasible for systems that demand lightweight operation, thereby impacting the practical deployment of VPR. To address this issue, we resort to knowledge distillation, which empowers single-modal students to learn from cross-modal teachers without introducing additional sensors during inference. Despite the notable advancements achieved by current distillation approaches, the exploration of feature relationships remains an under-explored area. In order to tackle the challenge of cross-modal distillation in VPR, we present DistilVPR, a novel distillation pipeline for VPR. We propose leveraging feature relationships from multiple agents, including self-agents and cross-agents for teacher and student neural networks. Furthermore, we integrate various manifolds, characterized by different space curvatures for exploring feature relationships. This approach enhances the diversity of feature relationships, including Euclidean, spherical, and hyperbolic relationship modules, thereby enhancing the overall representational capacity. The experiments demonstrate that our proposed pipeline achieves state-of-the-art performance compared to other distillation baselines. We also conduct necessary ablation studies to show design effectiveness. The code is released at: https://github.com/sijieaaa/DistilVPR