# Towards Content-based Pixel Retrieval in Revisited Oxford and Paris

arXiv: https://arxiv.org/abs/2309.05438 · HTML: https://mlsanity.com/paper/2309.05438/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 65 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 30 of 34)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [LiT: Zero-Shot Transfer with Locked-image text Tuning](https://mlsanity.com/paper/2111.07991/) (2021)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Simple Open-Vocabulary Object Detection with Vision Transformers](https://mlsanity.com/paper/2205.06230/) (2022)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [Deep Image Matting](https://mlsanity.com/paper/1703.03872/) (2017)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image](https://mlsanity.com/paper/2204.04752/) (2022)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [Mining Latent Classes for Few-shot Segmentation](https://mlsanity.com/paper/2103.15402/) (2021)
- [GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network](https://mlsanity.com/paper/2009.07823/) (2020)
- [Visual Instance Retrieval with Deep Convolutional Networks](https://mlsanity.com/paper/1412.6574/) (2014)
- [RANSAC-Flow: generic two-stage image alignment](https://mlsanity.com/paper/2004.01526/) (2020)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)

## Built on by (most cited first, 5 of 5)

- [Where's Waldo: Diffusion Features for Personalized Segmentation and Retrieval](https://mlsanity.com/paper/2405.18025/) (2024)
- [Neural Rendering and Its Hardware Acceleration: A Review](https://mlsanity.com/paper/2402.00028/) (2024)
- [Topological RANSAC for instance verification and retrieval without fine-tuning](https://mlsanity.com/paper/2310.06486/) (2023)
- [Accurate and Fast Pixel Retrieval with Spatial and Uncertainty Aware Hypergraph Diffusion](https://mlsanity.com/paper/2406.11242/) (2024)
- [Composed Object Retrieval: Object-level Retrieval via Composed Expressions](https://mlsanity.com/paper/2508.04424/) (2025)

## Abstract (arXiv)

This paper introduces the first two pixel retrieval benchmarks. Pixel retrieval is segmented instance retrieval. Like semantic segmentation extends classification to the pixel level, pixel retrieval is an extension of image retrieval and offers information about which pixels are related to the query object. In addition to retrieving images for the given query, it helps users quickly identify the query object in true positive images and exclude false positive images by denoting the correlated pixels. Our user study results show pixel-level annotation can significantly improve the user experience. Compared with semantic and instance segmentation, pixel retrieval requires a fine-grained recognition capability for variable-granularity targets. To this end, we propose pixel retrieval benchmarks named PROxford and PRParis, which are based on the widely used image retrieval datasets, ROxford and RParis. Three professional annotators label 5,942 images with two rounds of double-checking and refinement. Furthermore, we conduct extensive experiments and analysis on the SOTA methods in image search, image matching, detection, segmentation, and dense matching using our pixel retrieval benchmarks. Results show that the pixel retrieval task is challenging to these approaches and distinctive from existing problems, suggesting that further research can advance the content-based pixel-retrieval and thus user search experience. The datasets can be downloaded from hrefhttps://github.com/anguoyuan/Pixelretrieval-Segmentedinstanceretrievalthis link.