# StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video

arXiv: https://arxiv.org/abs/2305.00942 · HTML: https://mlsanity.com/paper/2305.00942/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 80 references that are in the map.
- First posted on arXiv 1 May 2023.

## Builds on (most cited first, 30 of 54)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering](https://mlsanity.com/paper/2109.08379/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN](https://mlsanity.com/paper/2203.04036/) (2022)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)

## Built on by (most cited first, 30 of 52)

- [Drivable 3D Gaussian Avatars](https://mlsanity.com/paper/2311.08581/) (2023)
- [3D Gaussian as a New Era: A Survey](https://mlsanity.com/paper/2402.07181/) (2024)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Pose](https://mlsanity.com/paper/2308.03610/) (2023)
- [LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar](https://mlsanity.com/paper/2305.01190/) (2023)
- [URAvatar: Universal Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2410.24223/) (2024)
- [GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations](https://mlsanity.com/paper/2409.11951/) (2024)
- [Animatable and Relightable Gaussians for High-fidelity Human Avatar Modeling](https://mlsanity.com/paper/2311.16096/) (2023)
- [LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer](https://mlsanity.com/paper/2405.07319/) (2024)
- [Gaussian Eigen Models for Human Heads](https://mlsanity.com/paper/2407.04545/) (2024)
- [TextToon: Real-Time Text Toonify Head Avatar from Single Video](https://mlsanity.com/paper/2410.07160/) (2024)
- [BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis](https://mlsanity.com/paper/2311.05521/) (2023)
- [InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars](https://mlsanity.com/paper/2312.02222/) (2023)
- [UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling](https://mlsanity.com/paper/2403.11589/) (2024)
- [Tri2-plane: Thinking Head Avatar via Feature Pyramid](https://mlsanity.com/paper/2401.09386/) (2024)
- [Synthetic Prior for Few-Shot Drivable Head Avatar Inversion](https://mlsanity.com/paper/2501.06903/) (2025)
- [TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.17032/) (2025)
- [Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes](https://mlsanity.com/paper/2404.01543/) (2024)
- [PICA: Physics-Integrated Clothed Avatar](https://mlsanity.com/paper/2407.05324/) (2024)
- [GaussianStyle: Gaussian Head Avatar via StyleGAN](https://mlsanity.com/paper/2402.00827/) (2024)
- [Towards Native Generative Model for 3D Head Avatar](https://mlsanity.com/paper/2410.01226/) (2024)
- [GaussianAvatar-Editor: Photorealistic Animatable Gaussian Head Avatar Editor](https://mlsanity.com/paper/2501.09978/) (2025)
- [InstaFace: Identity-Preserving Facial Editing with Single Image Inference](https://mlsanity.com/paper/2502.20577/) (2025)
- [GASPACHO: Gaussian Splatting for Controllable Humans and Objects](https://mlsanity.com/paper/2503.09342/) (2025)
- [Stable Video Portraits](https://mlsanity.com/paper/2409.18083/) (2024)
- [MyPortrait: Morphable Prior-Guided Personalized Portrait Generation](https://mlsanity.com/paper/2312.02703/) (2023)
- [Coherent 3D Portrait Video Reconstruction via Triplane Fusion](https://mlsanity.com/paper/2405.00794/) (2024)
- [Bringing Your Portrait to 3D Presence](https://mlsanity.com/paper/2511.22553/) (2025)
- [GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits](https://mlsanity.com/paper/2312.07669/) (2023)
- [Controllable and Expressive One-Shot Video Head Swapping](https://mlsanity.com/paper/2506.16852/) (2025)

## Abstract (arXiv)

Face reenactment methods attempt to restore and re-animate portrait videos as realistically as possible. Existing methods face a dilemma in quality versus controllability: 2D GAN-based methods achieve higher image quality but suffer in fine-grained control of facial attributes compared with 3D counterparts. In this work, we propose StyleAvatar, a real-time photo-realistic portrait avatar reconstruction method using StyleGAN-based networks, which can generate high-fidelity portrait avatars with faithful expression control. We expand the capabilities of StyleGAN by introducing a compositional representation and a sliding window augmentation method, which enable faster convergence and improve translation generalization. Specifically, we divide the portrait scenes into three parts for adaptive adjustments: facial region, non-facial foreground region, and the background. Besides, our network leverages the best of UNet, StyleGAN and time coding for video learning, which enables high-quality video generation. Furthermore, a sliding window augmentation method together with a pre-training strategy are proposed to improve translation generalization and training performance, respectively. The proposed network can converge within two hours while ensuring high image quality and a forward rendering time of only 20 milliseconds. Furthermore, we propose a real-time live system, which further pushes research into applications. Results and experiments demonstrate the superiority of our method in terms of image quality, full portrait video generation, and real-time re-animation compared to existing facial reenactment methods. Training and inference code for this paper are at https://github.com/LizhenWangT/StyleAvatar.