mlsanityData to 14 Oct 2026

Real-Time Radiance Fields for Single-Image Portrait View Synthesis

arXiv 2305.02310 ↗First posted 3 May 2023cs.CV cs.AI cs.GRcited by 64

Not on the map. Why?

Abstract (arXiv)

We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher quality results than strong GAN-inversion baselines that require test-time optimization. To train our triplane encoder pipeline, we use only synthetic data, showing how to distill the knowledge from a pretrained 3D GAN into a feedforward encoder. Technical contributions include a Vision Transformer-based triplane encoder, a camera data augmentation strategy, and a well-designed loss function for synthetic data training. We benchmark against the state-of-the-art methods, demonstrating significant improvements in robustness and image quality in challenging real-world settings. We showcase our results on portraits of faces (FFHQ) and cats (AFHQ), but our algorithm can also be applied in the future to other categories with a 3D-aware image generator.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 94 of its 119 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

94 of 119 references.

References · 94

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  10. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
38 more
  1. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  2. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  3. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  4. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  8. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  9. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  12. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  13. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  14. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  15. 2021Neural Fields in Visual Computing and Beyondcited by 540
  16. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  17. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  18. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  19. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  20. 2014Learning Face Representation from Scratchcited by 453
  21. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  22. 2015Unsupervised Learning of Video Representations using LSTMscited by 435
  23. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  24. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  25. 2021MagFace: A Universal Representation for Face Recognition and Quality Assessmentcited by 334
  26. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  27. 2022On the detection of synthetic images generated by diffusion modelscited by 283
  28. 2018Deep Video Portraitscited by 267
  29. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  30. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  31. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  32. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
  33. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  34. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 213
  35. 2022Thin-Plate Spline Motion Model for Image Animationcited by 210
  36. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 201
  37. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  38. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197

50 most cited of 94 All 94 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 35; 2025: 14; 2026: 7.

64 papers.

Sort:

By area:

  1. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  2. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  3. 2024GPAvatar: Generalizable and Precise Head Avatar from Image(s)cited by 74
  4. 2024Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizercited by 64
  5. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  6. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  7. 2024FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portraitcited by 40
  8. 2024Cafca: High-quality Novel View Synthesis of Expressive Faces from Casual Few-shot Capturescited by 34
  9. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  10. 2024GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representationscited by 31
  11. 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
  12. 2024Lite2Relight: 3D-aware Single Image Portrait Relightingcited by 25
38 more
  1. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creationcited by 23
  2. 2024Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solidscited by 23
  3. 2023Generalizable One-shot Neural Head Avatarcited by 21
  4. 2024Real-time 3D-aware Portrait Video Relightingcited by 21
  5. 2023BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesiscited by 20
  6. 2023InvertAvatar: Incremental GAN Inversion for Generalized Head Avatarscited by 19
  7. 2024VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresencecited by 19
  8. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 18
  9. 2024Tri2-plane: Thinking Head Avatar via Feature Pyramidcited by 16
  10. 2024Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modelingcited by 16
  11. 2024Quark: Real-time, High-resolution, and General Neural View Synthesiscited by 16
  12. 2024Learning to Generate Conditional Tri-plane for 3D-aware Expression Controllable Portrait Animationcited by 15
  13. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Priorcited by 15
  14. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  15. 2024One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptationcited by 14
  16. 2024What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANscited by 12
  17. 2025DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesiscited by 12
  18. 2024Tele-Aloha: A Low-budget and High-authenticity Telepresence System Using Sparse RGB Camerascited by 11
  19. 2024Real-time 3D-aware Portrait Editing from a Single Imagecited by 10
  20. 2023My3DGen: A Scalable Personalized 3D Generative Modelcited by 8
  21. 2023NeRFs: The Search for the Best 3D Representationcited by 7
  22. 2025FastAvatar: Instant 3D Gaussian Splatting for Faces from Single Unconstrained Posescited by 7
  23. 2025PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Imagecited by 7
  24. 2025Efficient Multi-Camera Tokenization with Triplanes for End-to-End Drivingcited by 6
  25. 2024SYM3D: Learning Symmetric Triplanes for Better 3D-Awareness of GANscited by 5
  26. 2026UIKA: Fast Universal Head Avatar from Pose-Free Imagescited by 5
  27. 2024MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Spacecited by 4
  28. 2024FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Modelcited by 4
  29. 2024Single Image, Any Face: Generalisable 3D Face Generationcited by 4
  30. 2025FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Imagescited by 4
  31. 2024G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Imagescited by 3
  32. 2024Generalizable Neural Human Renderercited by 3
  33. 2024Coherent 3D Portrait Video Reconstruction via Triplane Fusioncited by 3
  34. 2025SpinMeRound: Consistent Multi-View Identity Generation Using Diffusion Modelscited by 3
  35. 2025VoluMe -- Authentic 3D Video Calls from Live Gaussian Splat Predictioncited by 3
  36. 2025AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatarscited by 3
  37. 2026DiffHDR: Re-Exposing LDR Videos with Video Diffusion Modelscited by 3
  38. 2024Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Priorcited by 2

50 most cited of 64 All 64 (JSON)