mlsanityData to 14 Oct 2026

Video-Driven Animation of Neural Head Avatars

arXiv 2403.04380 ↗First posted 7 Mar 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a new approach for video-driven animation of high-quality neural 3D head models, addressing the challenge of person-independent animation from video input. Typically, high-quality generative models are learned for specific individuals from multi-view video footage, resulting in person-specific latent representations that drive the generation process. In order to achieve person-independent animation from video input, we introduce an LSTM-based animation network capable of translating person-independent expression features into personalized animation parameters of person-specific 3D head models. Our approach combines the advantages of personalized head models (high quality and realism) with the convenience of video-driven animation employing multi-person facial performance capture. We demonstrate the effectiveness of our approach on synthesized animations with high quality based on different source videos as well as an ablation study.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

31 of 54 references.

References · 31

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  5. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2020First Order Motion Model for Image Animationcited by 704
  8. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  9. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  10. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  11. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  12. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
19 more
  1. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  2. 2020State of the Art on Neural Renderingcited by 278
  3. 2018Deep Video Portraitscited by 267
  4. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 255
  5. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  6. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 178
  7. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
  8. 2018Deep Appearance Models for Face Renderingcited by 164
  9. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  10. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  11. 2021Pixel Codec Avatarscited by 127
  12. 2020Learning Formation of Physically-Based Face Attributescited by 94
  13. 2019MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targetscited by 90
  14. 2020SMPLpix: Neural Avatars from 3D Human Modelscited by 69
  15. 2022Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videoscited by 50
  16. 2023Face Animation with an Attribute-Guided Diffusion Modelcited by 48
  17. 2020Learning Complete 3D Morphable Face Models from Images and Videoscited by 35
  18. 2020Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Posecited by 26
  19. 2023Unsupervised Learning of Style-Aware Facial Animation from Real Acting Performancescited by 2

Built on by

Not cited yet.