# HyperInverter: Improving StyleGAN Inversion via Hypernetwork

arXiv: https://arxiv.org/abs/2112.00719 · HTML: https://mlsanity.com/paper/2112.00719/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 61 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)

## Built on by (most cited first, 30 of 85)

- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2302.12228/) (2023)
- [ReNoise: Real Image Inversion Through Iterative Noising](https://mlsanity.com/paper/2403.14602/) (2024)
- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- ["This is my unicorn, Fluffy": Personalizing frozen vision-language representations](https://mlsanity.com/paper/2204.01694/) (2022)
- [Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models](https://mlsanity.com/paper/2307.06925/) (2023)
- [Effective Real Image Editing with Accelerated Iterative Diffusion Inversion](https://mlsanity.com/paper/2309.04907/) (2023)
- [High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization](https://mlsanity.com/paper/2211.15662/) (2022)
- [Real-Time Radiance Fields for Single-Image Portrait View Synthesis](https://mlsanity.com/paper/2305.02310/) (2023)
- [TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models](https://mlsanity.com/paper/2408.00735/) (2024)
- [Third Time's the Charm? Image and Video Editing with StyleGAN3](https://mlsanity.com/paper/2201.13433/) (2022)
- [HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces](https://mlsanity.com/paper/2307.10797/) (2023)
- [3D GAN Inversion with Pose Optimization](https://mlsanity.com/paper/2210.07301/) (2022)
- [TriPlaneNet: An Encoder for EG3D Inversion](https://mlsanity.com/paper/2303.13497/) (2023)
- [DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization](https://mlsanity.com/paper/2209.05227/) (2022)
- [Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding](https://mlsanity.com/paper/2303.12326/) (2023)
- [State-of-the-Art in the Architecture, Methods and Applications of StyleGAN](https://mlsanity.com/paper/2202.14020/) (2022)
- [Idempotence and Perceptual Image Compression](https://mlsanity.com/paper/2401.08920/) (2024)
- [Finding Directions in GAN's Latent Space for Neural Face Reenactment](https://mlsanity.com/paper/2202.00046/) (2022)
- [Visual Instruction Inversion: Image Editing via Visual Prompting](https://mlsanity.com/paper/2307.14331/) (2023)
- [CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation](https://mlsanity.com/paper/2310.01407/) (2023)
- [High-fidelity GAN Inversion with Padding Space](https://mlsanity.com/paper/2203.11105/) (2022)
- [VToonify: Controllable High-Resolution Portrait Video Style Transfer](https://mlsanity.com/paper/2209.11224/) (2022)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces](https://mlsanity.com/paper/2303.06146/) (2023)
- [InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars](https://mlsanity.com/paper/2312.02222/) (2023)
- [Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint](https://mlsanity.com/paper/2211.11448/) (2022)

## Abstract (arXiv)

Real-world image manipulation has achieved fantastic progress in recent years as a result of the exploration and utilization of GAN latent spaces. GAN inversion is the first step in this pipeline, which aims to map the real image to the latent code faithfully. Unfortunately, the majority of existing GAN inversion methods fail to meet at least one of the three requirements listed below: high reconstruction quality, editability, and fast inference. We present a novel two-phase strategy in this research that fits all requirements at the same time. In the first phase, we train an encoder to map the input image to StyleGAN2 W-space, which was proven to have excellent editability but lower reconstruction quality. In the second phase, we supplement the reconstruction ability in the initial phase by leveraging a series of hypernetworks to recover the missing information during inversion. These two steps complement each other to yield high reconstruction quality thanks to the hypernetwork branch and excellent editability due to the inversion done in the W-space. Our method is entirely encoder-based, resulting in extremely fast inference. Extensive experiments on two challenging datasets demonstrate the superiority of our method.