# Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models

arXiv: https://arxiv.org/abs/1911.12287 · HTML: https://mlsanity.com/paper/1911.12287/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 21 of 21)

- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Is Attention Better Than Matrix Decomposition?](https://mlsanity.com/paper/2109.04553/) (2021)
- [Intermediate Layer Optimization for Inverse Problems using Deep Generative Models](https://mlsanity.com/paper/2102.07364/) (2021)
- [Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images](https://mlsanity.com/paper/2106.07873/) (2021)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Temporally Consistent Semantic Video Editing](https://mlsanity.com/paper/2206.10590/) (2022)
- [A survey on GANs for computer vision: Recent research, analysis and taxonomy](https://mlsanity.com/paper/2203.11242/) (2022)
- [Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art](https://mlsanity.com/paper/2308.16316/) (2023)
- [Comparing the latent space of generative models](https://mlsanity.com/paper/2207.06812/) (2022)
- [Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problems](https://mlsanity.com/paper/2206.09104/) (2022)
- [IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion](https://mlsanity.com/paper/2209.10811/) (2022)
- [Solving Inverse Problems with NerfGANs](https://mlsanity.com/paper/2112.09061/) (2021)
- [Image Embedding for Denoising Generative Models](https://mlsanity.com/paper/2301.07485/) (2022)
- [Simulating financial time series using attention](https://mlsanity.com/paper/2207.00493/) (2022)
- [In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing](https://mlsanity.com/paper/2302.04871/) (2023)
- [Variational Bayes image restoration with compressive autoencoders](https://mlsanity.com/paper/2311.17744/) (2023)
- [The Nuts and Bolts of Adopting Transformer in GANs](https://mlsanity.com/paper/2110.13107/) (2021)
- [A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration](https://mlsanity.com/paper/2207.10309/) (2022)
- [Face Morphing: Fooling a Face Recognition System Is Simple!](https://mlsanity.com/paper/2205.13796/) (2022)
- [PartStickers: Generating Parts of Objects for Rapid Prototyping](https://mlsanity.com/paper/2504.05508/) (2025)

## Abstract (arXiv)

We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FID, Inception score and pure visual improvements. FID score is improved from 18.65 to 15.94 on ImageNet, keeping all other parameters the same. The sparse attention patterns that we propose for our new layer are designed using a novel information theoretic criterion that uses information flow graphs. We also present a novel way to invert Generative Adversarial Networks with attention. Our method extracts from the attention layer of the discriminator a saliency map, which we use to construct a new loss function for the inversion. This allows us to visualize the newly introduced attention heads and show that they indeed capture interesting aspects of two-dimensional geometry of real images.