# REMAP: Multi-layer entropy-guided pooling of dense CNN features for image retrieval

arXiv: https://arxiv.org/abs/1906.06626 · HTML: https://mlsanity.com/paper/1906.06626/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2019.

## Built on by (most cited first, 9 of 9)

- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations](https://mlsanity.com/paper/2209.14905/) (2022)
- [A Critical Analysis of Image-based Camera Pose Estimation Techniques](https://mlsanity.com/paper/2201.05816/) (2022)
- [Content-Based Medical Image Retrieval with Opponent Class Adaptive Margin Loss](https://mlsanity.com/paper/2211.15371/) (2022)
- [EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud Computing](https://mlsanity.com/paper/2208.14657/) (2022)
- [Adversarial Learning of Hard Positives for Place Recognition](https://mlsanity.com/paper/2205.03871/) (2022)
- [Understanding the Distributions of Aggregation Layers in Deep Neural Networks](https://mlsanity.com/paper/2107.04458/) (2021)
- [A Recipe for Efficient SBIR Models: Combining Relative Triplet Loss with Batch Normalization and Knowledge Distillation](https://mlsanity.com/paper/2305.18988/) (2023)

## Abstract (arXiv)

This paper addresses the problem of very large-scale image retrieval, focusing on improving its accuracy and robustness. We target enhanced robustness of search to factors such as variations in illumination, object appearance and scale, partial occlusions, and cluttered backgrounds - particularly important when search is performed across very large datasets with significant variability. We propose a novel CNN-based global descriptor, called REMAP, which learns and aggregates a hierarchy of deep features from multiple CNN layers, and is trained end-to-end with a triplet loss. REMAP explicitly learns discriminative features which are mutually-supportive and complementary at various semantic levels of visual abstraction. These dense local features are max-pooled spatially at each layer, within multi-scale overlapping regions, before aggregation into a single image-level descriptor. To identify the semantically useful regions and layers for retrieval, we propose to measure the information gain of each region and layer using KL-divergence. Our system effectively learns during training how useful various regions and layers are and weights them accordingly. We show that such relative entropy-guided aggregation outperforms classical CNN-based aggregation controlled by SGD. The entire framework is trained in an end-to-end fashion, outperforming the latest state-of-the-art results. On image retrieval datasets Holidays, Oxford and MPEG, the REMAP descriptor achieves mAP of 95.5%, 91.5%, and 80.1% respectively, outperforming any results published to date. REMAP also formed the core of the winning submission to the Google Landmark Retrieval Challenge on Kaggle.