mlsanityData to 7 Oct 2026

Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions

arXiv 2209.04436 ↗First posted 7 Sep 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation unicodex2013 neural implicit functions unicodex2013 which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 7 Sep 2022.

Builds on

43 of 60 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  12. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
31 more
  1. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  2. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  3. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  4. 2016Learning to learn by gradient descent by gradient descentcited by 616
  5. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  6. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  7. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  8. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  9. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 240
  10. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  11. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  12. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  13. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  14. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  15. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  16. 2021Neural Lumigraph Renderingcited by 105
  17. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  18. 2019Graphonomy: Universal Human Parsing via Graph Transfer Learningcited by 81
  19. 2020PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representationscited by 81
  20. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
  21. 2017Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regressioncited by 66
  22. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  23. 2020Portrait Neural Radiance Fields from a Single Imagecited by 61
  24. 2019Unsupervised Generative 3D Shape Learning from Natural Imagescited by 57
  25. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  26. 2020High-Fidelity 3D Digital Human Head Creation from RGB-D Selfiescited by 47
  27. 2020Deep 3D Portrait from a Single Imagecited by 38
  28. 20173D Face Reconstruction with Geometry Details from a Single Imagecited by 27
  29. 2020Survey on 3D face reconstruction from uncalibrated imagescited by 24
  30. 2019'Zhores' -- Petaflops supercomputer for data-driven modeling, machine learning and artificial intelligence installed in Skolkovo Institute of Science and Technologycited by 23
  31. 2022SmartPortraits: Depth Powered Handheld Smartphone Dataset of Human Portraits for State Estimation, Reconstruction and Synthesiscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 1.

6 papers.

  1. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creationcited by 23
  2. 2022PhoMoH: Implicit Photorealistic 3D Models of Human Headscited by 7
  3. 2023InstantAvatar: Efficient 3D Head Reconstruction via Surface Renderingcited by 4
  4. 2023Head3D: Complete 3D Head Generation via Tri-plane Feature Distillationcited by 3
  5. 2023SPHEAR: Spherical Head Registration for Complete Statistical 3D Modelingcited by 0
  6. 20253D Shape Generation: A Surveycited by 0