mlsanityData to 7 Oct 2026

Implicit Neural Deformation for Sparse-View Face Reconstruction

arXiv 2112.02494 ↗First posted 5 Dec 2021cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we present a new method for 3D face reconstruction from sparse-view RGB images. Unlike previous methods which are built upon 3D morphable models (3DMMs) with limited details, we leverage an implicit representation to encode rich geometric features. Our overall pipeline consists of two major components, including a geometry network, which learns a deformable neural signed distance function (SDF) as the 3D face representation, and a rendering network, which learns to render on-surface points of the neural SDF to match the input images via self-supervised optimization. To handle in-the-wild sparse-view input of the same target with different expressions at test time, we propose residual latent code to effectively expand the shape space of the learned implicit face representation as well as a novel view-switch loss to enforce consistency among different views. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state-of-the-art methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 70 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

43 of 70 references.

References · 43

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  3. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  5. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  6. 2020Neural Sparse Voxel Fieldscited by 959
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  10. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  11. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  12. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
31 more
  1. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  2. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  3. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  4. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  5. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  6. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  7. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  8. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  9. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 179
  10. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  11. 2020DeRF: Decomposed Radiance Fieldscited by 163
  12. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  13. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  14. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  15. 2021Neural Lumigraph Renderingcited by 105
  16. 2020Deep Implicit Templates for 3D Shape Representationcited by 86
  17. 2018FML: Face Model Learning from Videoscited by 78
  18. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  19. 2016Learning Detailed Face Reconstruction from a Single Imagecited by 71
  20. 2020Curriculum DeepSDFcited by 62
  21. 2020Portrait Neural Radiance Fields from a Single Imagecited by 61
  22. 2020A Morphable Face Albedo Modelcited by 54
  23. 2017End-to-end 3D face reconstruction with deep neural networkscited by 47
  24. 2019MVF-Net: Multi-View 3D Face Morphable Model Regressioncited by 45
  25. 2019Photo-Realistic Facial Details Synthesis from Single Imagecited by 40
  26. 2020Deep 3D Portrait from a Single Imagecited by 38
  27. 2021NeLF: Neural Light-transport Field for Portrait View Synthesis and Relightingcited by 38
  28. 2021Riggable 3D Face Reconstruction via In-Network Optimizationcited by 31
  29. 2020On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapescited by 29
  30. 2021OctField: Hierarchical Implicit Functions for 3D Modelingcited by 25
  31. 2021DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Controlcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2025FatesGS: Fast and Accurate Sparse-View Surface Reconstruction using Gaussian Splatting with Depth-Feature Consistencycited by 30
  2. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  3. 2023Deformable Model-Driven Neural Rendering for High-Fidelity 3D Reconstruction of Human Heads Under Low-View Settingscited by 7
  4. 2024Democratizing the Creation of Animatable Facial Avatarscited by 1