# Pattern Detection in the Activation Space for Identifying Synthesized Content

arXiv: https://arxiv.org/abs/2105.12479 · HTML: https://mlsanity.com/paper/2105.12479/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [MedGAN: Medical Image Translation using GANs](https://mlsanity.com/paper/1806.06397/) (2018)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text](https://mlsanity.com/paper/1802.01345/) (2018)
- [Adversarial Text Generation via Feature-Mover's Distance](https://mlsanity.com/paper/1809.06297/) (2018)
- [Learning to Detect Fake Face Images in the Wild](https://mlsanity.com/paper/1809.08754/) (2018)
- [Identifying Audio Adversarial Examples via Anomalous Pattern Detection](https://mlsanity.com/paper/2002.05463/) (2020)
- [Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama's voice using GAN, WaveNet and low-quality found data](https://mlsanity.com/paper/1803.00860/) (2018)

## Built on by (most cited first, 5 of 5)

- [Improving Interpretability and Robustness for the Detection of AI-Generated Images](https://mlsanity.com/paper/2406.15035/) (2024)
- [Localizing Persona Representations in LLMs](https://mlsanity.com/paper/2505.24539/) (2025)
- [Efficient Representation of the Activation Space in Deep Neural Networks](https://mlsanity.com/paper/2312.08143/) (2023)
- [Sparsity-based Feature Selection for Anomalous Subgroup Discovery](https://mlsanity.com/paper/2201.02008/) (2022)
- [Analyzing Decades-Long Environmental Changes in Namibia Using Archival Aerial Photography and Deep Learning](https://mlsanity.com/paper/2404.08544/) (2024)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have recently achieved unprecedented success in photo-realistic image synthesis from low-dimensional random noise. The ability to synthesize high-quality content at a large scale brings potential risks as the generated samples may lead to misinformation that can create severe social, political, health, and business hazards. We propose SubsetGAN to identify generated content by detecting a subset of anomalous node-activations in the inner layers of pre-trained neural networks. These nodes, as a group, maximize a non-parametric measure of divergence away from the expected distribution of activations created from real data. This enable us to identify synthesised images without prior knowledge of their distribution. SubsetGAN efficiently scores subsets of nodes and returns the group of nodes within the pre-trained classifier that contributed to the maximum score. The classifier can be a general fake classifier trained over samples from multiple sources or the discriminator network from different GANs. Our approach shows consistently higher detection power than existing detection methods across several state-of-the-art GANs (PGGAN, StarGAN, and CycleGAN) and over different proportions of generated content.