# Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images

arXiv: https://arxiv.org/abs/1512.02017 · HTML: https://mlsanity.com/paper/1512.02017/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2015.

## Built on by (most cited first, 30 of 93)

- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?](https://mlsanity.com/paper/2111.01124/) (2021)
- [GradViT: Gradient Inversion of Vision Transformers](https://mlsanity.com/paper/2203.11894/) (2022)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [Optimizing Relevance Maps of Vision Transformers Improves Robustness](https://mlsanity.com/paper/2206.01161/) (2022)
- [Explain Any Concept: Segment Anything Meets Concept-Based Explanation](https://mlsanity.com/paper/2305.10289/) (2023)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF](https://mlsanity.com/paper/2202.11479/) (2022)
- [Discovering Distinctive "Semantics" in Super-Resolution Networks](https://mlsanity.com/paper/2108.00406/) (2021)
- [Visual Explanations via Iterated Integrated Attributions](https://mlsanity.com/paper/2310.18585/) (2023)
- [Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer](https://mlsanity.com/paper/2403.14552/) (2024)
- [On the Faithfulness of Vision Transformer Explanations](https://mlsanity.com/paper/2404.01415/) (2024)
- [Symbolic regression for scientific discovery: an application to wind speed forecasting](https://mlsanity.com/paper/2102.10570/) (2021)
- [Explainable Machine Learning with Prior Knowledge: An Overview](https://mlsanity.com/paper/2105.10172/) (2021)
- [Saliency Guided Experience Packing for Replay in Continual Learning](https://mlsanity.com/paper/2109.04954/) (2021)
- [Designing Counterfactual Generators using Deep Model Inversion](https://mlsanity.com/paper/2109.14274/) (2021)
- [Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing](https://mlsanity.com/paper/2112.08796/) (2021)
- [Core Risk Minimization using Salient ImageNet](https://mlsanity.com/paper/2203.15566/) (2022)
- [A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks](https://mlsanity.com/paper/2102.01792/) (2021)
- [Counterfactual Explanations for Oblique Decision Trees: Exact, Efficient Algorithms](https://mlsanity.com/paper/2103.01096/) (2021)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [Reconstructing Training Data From Real World Models Trained with Transfer Learning](https://mlsanity.com/paper/2407.15845/) (2024)
- [Deep Integrated Explanations](https://mlsanity.com/paper/2310.15368/) (2023)
- [GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps](https://mlsanity.com/paper/2109.00951/) (2021)
- [Quantum Gradient Class Activation Map for Model Interpretability](https://mlsanity.com/paper/2408.05899/) (2024)
- [Visualizing Global Explanations of Point Cloud DNNs](https://mlsanity.com/paper/2203.09505/) (2022)
- [Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing](https://mlsanity.com/paper/2209.03268/) (2022)
- [Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval](https://mlsanity.com/paper/2308.04343/) (2023)

## Abstract (arXiv)

Image representations, from SIFT and bag of visual words to Convolutional Neural Networks (CNNs) are a crucial component of almost all computer vision systems. However, our understanding of them remains limited. In this paper we study several landmark representations, both shallow and deep, by a number of complementary visualization techniques. These visualizations are based on the concept of "natural pre-image", namely a natural-looking image whose representation has some notable property. We study in particular three such visualizations: inversion, in which the aim is to reconstruct an image from its representation, activation maximization, in which we search for patterns that maximally stimulate a representation component, and caricaturization, in which the visual patterns that a representation detects in an image are exaggerated. We pose these as a regularized energy-minimization framework and demonstrate its generality and effectiveness. In particular, we show that this method can invert representations such as HOG more accurately than recent alternatives while being applicable to CNNs too. Among our findings, we show that several layers in CNNs retain photographically accurate information about the image, with different degrees of geometric and photometric invariance.