mlsanityData to 14 Oct 2026

GAN-Supervised Dense Visual Alignment

arXiv 2112.05143 ↗First posted 9 Dec 2021cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

We propose GAN-Supervised Learning, a framework for learning discriminative models and their GAN-generated training data jointly end-to-end. We apply our framework to the dense visual alignment problem. Inspired by the classic Congealing method, our GANgealing algorithm trains a Spatial Transformer to map random samples from a GAN trained on unaligned data to a common, jointly-learned target mode. We show results on eight datasets, all of which demonstrate our method successfully aligns complex data and discovers dense correspondences. GANgealing significantly outperforms past self-supervised correspondence algorithms and performs on-par with (and sometimes exceeds) state-of-the-art supervised correspondence algorithms on several datasets -- without making use of any correspondence supervision or data augmentation and despite being trained exclusively on GAN-generated data. For precise correspondence, we improve upon state-of-the-art supervised methods by as much as 3×. We show applications of our method for augmented reality, image editing and automated pre-processing of image datasets for downstream GAN training.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 72 of its 97 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

72 of 97 references.

References · 72

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
38 more
  1. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  2. 2017Deformable Convolutional Networkscited by 1,488
  3. 2015Spatial Transformer Networkscited by 1,429
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  6. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  9. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  10. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  11. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  12. 2016Adversarial Feature Learningcited by 350
  13. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  14. 2016Dynamic Filter Networkscited by 319
  15. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  16. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  17. 2019Large Scale Adversarial Representation Learningcited by 239
  18. 2016Adversarially Learned Inferencecited by 216
  19. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  20. 2018Neighbourhood Consensus Networkscited by 174
  21. 2016View Synthesis by Appearance Flowcited by 172
  22. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  23. 2019Seeing What a GAN Cannot Generatecited by 148
  24. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 146
  25. 2017Convolutional neural network architecture for geometric matchingcited by 138
  26. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  27. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  28. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  29. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  30. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  31. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  32. 2021Learning to See by Looking at Noisecited by 92
  33. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  34. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  35. 2020Articulation-aware Canonical Surface Mappingcited by 81
  36. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  37. 2020Transforming and Projecting Images into Class-conditional Generative Networkscited by 76
  38. 2019This dataset does not exist: training models from generated imagescited by 65

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 17; 2023: 20; 2024: 13; 2025: 6; 2026: 6.

62 papers.

Sort:

By area:

  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  2. 2023Emergent Correspondence from Image Diffusioncited by 446
  3. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  4. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  5. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  6. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  7. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  8. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
  9. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  10. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  11. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  12. 2022CATs++: Boosting Cost Aggregation with Convolutions and Transformerscited by 53
38 more
  1. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 38
  2. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  3. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  4. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 31
  5. 2023ASIC: Aligning Sparse in-the-wild Image Collectionscited by 27
  6. 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
  7. 2022Region-Based Semantic Factorization in GANscited by 22
  8. 2022Improving GANs with A Dynamic Discriminatorcited by 20
  9. 2023CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Imagescited by 19
  10. 2022CoordGAN: Self-Supervised Dense Correspondences Emerge from GANscited by 17
  11. 2025MangaNinja: Line Art Colorization with Precise Reference Followingcited by 17
  12. 2023Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Cameracited by 16
  13. 2024Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondencecited by 14
  14. 2024SHIC: Shape-Image Correspondences with no Keypoint Supervisioncited by 12
  15. 2024SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimationcited by 12
  16. 2023Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generationcited by 11
  17. 2024Zero-Shot Image Feature Consensus with Deep Functional Mapscited by 10
  18. 2025Follow-Your-Color: Multi-Instance Sketch Colorizationcited by 10
  19. 2022Video2StyleGAN: Disentangling Local and Global Variations in a Videocited by 8
  20. 2022Correspondence Distillation from NeRF-based GANcited by 8
  21. 2023POCE: Pose-Controllable Expression Editingcited by 8
  22. 2023GeNIe: Generative Hard Negative Images Through Diffusioncited by 8
  23. 2022Content-Based Search for Deep Generative Modelscited by 7
  24. 2023PG-VTON: A Novel Image-Based Virtual Try-On Method via Progressive Inference Paradigmcited by 7
  25. 2025PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Imagecited by 7
  26. 2022Integrative Feature and Cost Aggregation with Transformers for Dense Correspondencecited by 6
  27. 2023Efficient Semantic Matching with Hypercolumn Correlationcited by 6
  28. 2024SpaceJAM: a Lightweight and Regularization-free Method for Fast Joint Alignment of Imagescited by 6
  29. 2023Hashing Neural Video Decomposition with Multiplicative Residuals in Space-Timecited by 5
  30. 2024Controllable Image Synthesis of Industrial Data Using Stable Diffusioncited by 5
  31. 2024Edit One for All: Interactive Batch Image Editingcited by 5
  32. 2022NeuralMarker: A Framework for Learning General Marker Correspondencecited by 4
  33. 2024Stylized Face Sketch Extraction via Generative Prior with Limited Datacited by 4
  34. 2024Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detectioncited by 2
  35. 2022Interpreting Class Conditional GANs with Channel Awarenesscited by 1
  36. 2022A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restorationcited by 1
  37. 2022S2-Flow: Joint Semantic and Style Editing of Facial Imagescited by 1
  38. 2022Brain Tumor Synthetic Data Generation with Adaptive StyleGANscited by 1

50 most cited of 62 All 62 (JSON)