# Privacy Preserving Visual Question Answering

arXiv: https://arxiv.org/abs/2202.07712 · HTML: https://mlsanity.com/paper/2202.07712/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017)
- [In Defense of Grid Features for Visual Question Answering](https://mlsanity.com/paper/2001.03615/) (2020)
- [Learning to Reason: End-to-End Module Networks for Visual Question Answering](https://mlsanity.com/paper/1704.05526/) (2017)
- [Visual Question Answering: A Survey of Methods and Datasets](https://mlsanity.com/paper/1607.05910/) (2016)
- [Pythia v0.1: the Winning Entry to the VQA Challenge 2018](https://mlsanity.com/paper/1807.09956/) (2018)
- [Language-Conditioned Graph Networks for Relational Reasoning](https://mlsanity.com/paper/1905.04405/) (2019)
- [MiniVLM: A Smaller and Faster Vision-Language Model](https://mlsanity.com/paper/2012.06946/) (2020)
- [Deep Multimodal Neural Architecture Search](https://mlsanity.com/paper/2004.12070/) (2020)
- [Bilinear Graph Networks for Visual Question Answering](https://mlsanity.com/paper/1907.09815/) (2019)

## Built on by (most cited first, 2 of 2)

- [Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training](https://mlsanity.com/paper/2306.08173/) (2023)
- [ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning](https://mlsanity.com/paper/2610.01184/) (2026)

## Abstract (arXiv)

We introduce a novel privacy-preserving methodology for performing Visual Question Answering on the edge. Our method constructs a symbolic representation of the visual scene, using a low-complexity computer vision model that jointly predicts classes, attributes and predicates. This symbolic representation is non-differentiable, which means it cannot be used to recover the original image, thereby keeping the original image private. Our proposed hybrid solution uses a vision model which is more than 25 times smaller than the current state-of-the-art (SOTA) vision models, and 100 times smaller than end-to-end SOTA VQA models. We report detailed error analysis and discuss the trade-offs of using a distilled vision model and a symbolic representation of the visual scene.