mlsanityData to 7 Oct 2026

Pixel Codec Avatars

arXiv 2104.04638 ↗First posted 9 Apr 2021cs.CVcited by 125

Not on the map. Why?

Abstract (arXiv)

Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical distances. In this work, we present the Pixel Codec Avatars (PiCA): a deep generative model of 3D human faces that achieves state of the art reconstruction performance while being computationally efficient and adaptive to the rendering conditions during execution. Our model combines two core ideas: (1) a fully convolutional architecture for decoding spatially varying features, and (2) a rendering-adaptive per-pixel decoder. Both techniques are integrated via a dense surface representation that is learned in a weakly-supervised manner from low-topology mesh tracking over training images. We demonstrate that PiCA improves reconstruction over existing techniques across testing expressions and views on persons of different gender and skin tone. Importantly, we show that the PiCA model is much smaller than the state-of-art baseline model, and makes multi-person telecommunicaiton possible: on a single Oculus Quest 2 mobile VR headset, 5 avatars are rendered in realtime in the same scene.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Builds on 15 of its 28 references that are in the map. First posted on arXiv 9 Apr 2021.

Builds on

15 of 28 references.

References · 15

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2019Neural Point-Based Graphicscited by 326
  5. 2020State of the Art on Neural Renderingcited by 276
  6. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  7. 2018Deep Appearance Models for Face Renderingcited by 164
  8. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  9. 2018Nonlinear 3D Face Morphable Modelcited by 157
  10. 2020Audio- and Gaze-driven Facial Animation of Codec Avatarscited by 46
  11. 2019MeshGAN: Non-linear 3D Morphable Models of Facescited by 39
  12. 2018High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networkscited by 22
3 more
  1. 2019Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoderscited by 22
  2. 2020Expressive Telepresence via Modular Codec Avatarscited by 17
  3. 2019Synthesizing facial photometries and corresponding geometries using generative adversarial networkscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 14; 2023: 27; 2024: 35; 2025: 27; 2026: 19.

125 papers.

Sort:

By area:

  1. 2021Neural Fields in Visual Computing and Beyondcited by 539
  2. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  3. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  4. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  5. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 171
  6. 2023Relightable Gaussian Codec Avatarscited by 126
  7. 2021The Power of Points for Modeling Humans in Clothingcited by 90
  8. 2023MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatarcited by 89
  9. 2022Multiface: A Dataset for Neural Face Renderingcited by 87
  10. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  11. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
  12. 2022P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Promptingcited by 74
38 more
  1. 2024Advances in 3D Generation: A Surveycited by 62
  2. 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
  3. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  4. 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
  5. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  6. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  7. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  8. 2022UV Volumes for Real-time Rendering of Editable Free-view Human Performancecited by 43
  9. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  10. 2023HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Fieldcited by 37
  11. 2024MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editingcited by 37
  12. 2023HHAvatar: Gaussian Head Avatar with Dynamic Hairscited by 35
  13. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  14. 2022REALY: Rethinking the Evaluation of 3D Face Reconstructioncited by 32
  15. 2024GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representationscited by 31
  16. 2023FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learningcited by 29
  17. 2024MegActor: Harness the Power of Raw Video for Vivid Portrait Animationcited by 27
  18. 2022PV3D: A 3D Generative Model for Portrait Video Generationcited by 23
  19. 2024PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshingcited by 23
  20. 2024Gaussian Eigen Models for Human Headscited by 21
  21. 2024VRMM: A Volumetric Relightable Morphable Head Modelcited by 19
  22. 2024VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresencecited by 19
  23. 2022Video-driven Neural Physically-based Facial Asset for Productioncited by 17
  24. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 17
  25. 2022Neural Parameterization for Dynamic Human Head Editingcited by 16
  26. 2024Quark: Real-time, High-resolution, and General Neural View Synthesiscited by 16
  27. 2025MagicInfinite: Generating Infinite Talking Videos with Your Words and Voicecited by 16
  28. 2022MonoNHR: Monocular Neural Human Renderercited by 15
  29. 2023Animatable 3D Gaussians for High-fidelity Synthesis of Human Motionscited by 15
  30. 2024SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatarscited by 15
  31. 2023Implicit Neural Head Synthesis via Controllable Local Deformation Fieldscited by 14
  32. 2023Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatarcited by 14
  33. 2023GAN-Avatar: Controllable Personalized GAN-based Human Head Avatarcited by 14
  34. 2024Rasterized Edge Gradients: Handling Discontinuities Differentiablycited by 14
  35. 2025Synthetic Prior for Few-Shot Drivable Head Avatar Inversioncited by 14
  36. 2024MegActor-Sigma: Unlocking Flexible Mixed-Modal Control in Portrait Animation with Diffusion Transformercited by 13
  37. 2025MixerMDM: Learnable Composition of Human Motion Diffusion Modelscited by 13
  38. 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformerscited by 13

50 most cited of 125 All 125 (JSON)