# Differentiable Product Quantization for Memory Efficient Camera Relocalization

arXiv: https://arxiv.org/abs/2407.15540 · HTML: https://mlsanity.com/paper/2407.15540/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 81 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors](https://mlsanity.com/paper/2103.17265/) (2021)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses](https://mlsanity.com/paper/2305.14059/) (2023)
- [LaMAR: Benchmarking Localization and Mapping for Augmented Reality](https://mlsanity.com/paper/2210.10770/) (2022)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching](https://mlsanity.com/paper/2204.11700/) (2022)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [GLACE: Global Local Accelerated Coordinate Encoding](https://mlsanity.com/paper/2406.04340/) (2024)

## Built on by (most cited first, 9 of 9)

- [R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization](https://mlsanity.com/paper/2501.01421/) (2025)
- [GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations](https://mlsanity.com/paper/2508.18242/) (2025)
- [A2-GNN: Angle-Annular GNN for Visual Descriptor-free Camera Relocalization](https://mlsanity.com/paper/2502.20036/) (2025)
- [ImLoc: Revisiting Visual Localization with Image-based Representation](https://mlsanity.com/paper/2601.04185/) (2026)
- [Understanding and Optimizing Attention-Based Sparse Matching for Diverse Local Features](https://mlsanity.com/paper/2602.08430/) (2026)
- [FUSELOC: Fusing Global and Local Descriptors to Disambiguate 2D-3D Matching in Visual Localization](https://mlsanity.com/paper/2408.12037/) (2024)
- [Robust Scene Coordinate Regression via Geometrically-Consistent Global Descriptors](https://mlsanity.com/paper/2512.17226/) (2025)
- [L3:Scene-agnostic Visual Localization in the Wild](https://mlsanity.com/paper/2603.07937/) (2026)
- [GeoMix: Descriptor-Free Visual Localization via Global Context and Multi-Detector Training](https://mlsanity.com/paper/2607.02486/) (2026)

## Abstract (arXiv)

Camera relocalization relies on 3D models of the scene with a large memory footprint that is incompatible with the memory budget of several applications. One solution to reduce the scene memory size is map compression by removing certain 3D points and descriptor quantization. This achieves high compression but leads to performance drop due to information loss. To address the memory performance trade-off, we train a light-weight scene-specific auto-encoder network that performs descriptor quantization-dequantization in an end-to-end differentiable manner updating both product quantization centroids and network parameters through back-propagation. In addition to optimizing the network for descriptor reconstruction, we encourage it to preserve the descriptor-matching performance with margin-based metric loss functions. Results show that for a local descriptor memory of only 1MB, the synergistic combination of the proposed network and map compression achieves the best performance on the Aachen Day-Night compared to existing compression methods.