# Large Scale Adversarial Representation Learning

arXiv: https://arxiv.org/abs/1907.02544 · HTML: https://mlsanity.com/paper/1907.02544/

## Key facts

- Cited by 239 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2019.

## Built on by (most cited first, 30 of 239)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis](https://mlsanity.com/paper/2211.09117/) (2022)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [DetCo: Unsupervised Contrastive Learning for Object Detection](https://mlsanity.com/paper/2102.04803/) (2021)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [A Good Image Generator Is What You Need for High-Resolution Video Synthesis](https://mlsanity.com/paper/2104.15069/) (2021)
- [Efficient Self-supervised Vision Transformers for Representation Learning](https://mlsanity.com/paper/2106.09785/) (2021)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [Denoising Diffusion Autoencoders are Unified Self-supervised Learners](https://mlsanity.com/paper/2303.09769/) (2023)
- [Generative Models as a Data Source for Multiview Representation Learning](https://mlsanity.com/paper/2106.05258/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [Instance Localization for Self-supervised Detection Pretraining](https://mlsanity.com/paper/2102.08318/) (2021)
- [SiT: Self-supervised vIsion Transformer](https://mlsanity.com/paper/2104.03602/) (2021)
- [Towards the Generalization of Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2111.00743/) (2021)
- [Return of Unconditional Generation: A Self-supervised Representation Generation Method](https://mlsanity.com/paper/2312.03701/) (2023)
- [Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing](https://mlsanity.com/paper/2104.14754/) (2021)
- [Deconstructing Denoising Diffusion Models for Self-Supervised Learning](https://mlsanity.com/paper/2401.14404/) (2024)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [Divide and Contrast: Self-supervised Learning from Uncurated Data](https://mlsanity.com/paper/2105.08054/) (2021)
- [ReSSL: Relational Self-Supervised Learning with Weak Augmentation](https://mlsanity.com/paper/2107.09282/) (2021)
- [Weakly Supervised Contrastive Learning](https://mlsanity.com/paper/2110.04770/) (2021)

## Abstract (arXiv)

Adversarially trained generative models (GANs) have recently achieved compelling image synthesis results. But despite early successes in using GANs for unsupervised representation learning, they have since been superseded by approaches based on self-supervision. In this work we show that progress in image generation quality translates to substantially improved representation learning performance. Our approach, BigBiGAN, builds upon the state-of-the-art BigGAN model, extending it to representation learning by adding an encoder and modifying the discriminator. We extensively evaluate the representation learning and generation capabilities of these BigBiGAN models, demonstrating that these generation-based models achieve the state of the art in unsupervised representation learning on ImageNet, as well as in unconditional image generation. Pretrained BigBiGAN models -- including image generators and encoders -- are available on TensorFlow Hub (https://tfhub.dev/s?publisher=deepmind&q=bigbigan).