# Analysis and Mitigations of Reverse Engineering Attacks on Local Feature Descriptors

arXiv: https://arxiv.org/abs/2105.03812 · HTML: https://mlsanity.com/paper/2105.03812/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 9 May 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Inverting Visual Representations with Convolutional Networks](https://mlsanity.com/paper/1506.02753/) (2015)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [Revealing Scenes by Inverting Structure from Motion Reconstructions](https://mlsanity.com/paper/1904.03303/) (2019)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study](https://mlsanity.com/paper/1807.08379/) (2018)
- [Privacy Preserving Image-Based Localization](https://mlsanity.com/paper/1903.05572/) (2019)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [Learning Privacy Preserving Encodings through Adversarial Training](https://mlsanity.com/paper/1802.05214/) (2018)
- [Privacy-Preserving Action Recognition using Coded Aperture Videos](https://mlsanity.com/paper/1902.09085/) (2019)
- [Privacy Preserving Visual SLAM](https://mlsanity.com/paper/2007.10361/) (2020)
- [Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption](https://mlsanity.com/paper/1704.02203/) (2017)
- [Image Reconstruction from Bag-of-Visual-Words](https://mlsanity.com/paper/1505.05190/) (2015)
- [Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings](https://mlsanity.com/paper/2006.06634/) (2020)

## Built on by (most cited first, 10 of 10)

- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [Fully Geometric Panoramic Localization](https://mlsanity.com/paper/2403.19904/) (2024)
- [Privacy-Preserving Representations are not Enough -- Recovering Scene Content from Camera Poses](https://mlsanity.com/paper/2305.04603/) (2023)
- [Privacy-Preserving Visual Localization with Event Cameras](https://mlsanity.com/paper/2212.03177/) (2022)
- [Obfuscation Based Privacy Preserving Representations are Recoverable Using Neighborhood Information](https://mlsanity.com/paper/2409.11536/) (2024)
- [PS-FedGAN: An Efficient Federated Learning Framework Based on Partially Shared Generative Adversarial Networks For Data Privacy](https://mlsanity.com/paper/2305.11437/) (2023)
- [LDL: Line Distance Functions for Panoramic Localization](https://mlsanity.com/paper/2308.13989/) (2023)
- [UFed-GAN: A Secure Federated Learning Framework with Constrained Computation and Unlabeled Data](https://mlsanity.com/paper/2308.05870/) (2023)
- [A Distributed Framework for Privacy-Enhanced Vision Transformers on the Edge](https://mlsanity.com/paper/2512.09309/) (2025)
- [Revisiting Geometric Obfuscation with Dual Convergent Lines for Privacy-Preserving Image Queries in Visual Localization](https://mlsanity.com/paper/2604.22310/) (2026)

## Abstract (arXiv)

As autonomous driving and augmented reality evolve, a practical concern is data privacy. In particular, these applications rely on localization based on user images. The widely adopted technology uses local feature descriptors, which are derived from the images and it was long thought that they could not be reverted back. However, recent work has demonstrated that under certain conditions reverse engineering attacks are possible and allow an adversary to reconstruct RGB images. This poses a potential risk to user privacy. We take this a step further and model potential adversaries using a privacy threat model. Subsequently, we show under controlled conditions a reverse engineering attack on sparse feature maps and analyze the vulnerability of popular descriptors including FREAK, SIFT and SOSNet. Finally, we evaluate potential mitigation techniques that select a subset of descriptors to carefully balance privacy reconstruction risk while preserving image matching accuracy; our results show that similar accuracy can be obtained when revealing less information.