mlsanityData to 7 Oct 2026

HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video

arXiv 2201.04127 ↗First posted 11 Jan 2022cs.CV cs.GRcited by 483

Not on the map. Why?

Cited by 34 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a free-viewpoint rendering method -- HumanNeRF -- that works on a given monocular video of a human performing complex body motions, e.g. a video from YouTube. Our method enables pausing the video at any frame and rendering the subject from arbitrary new camera viewpoints or even a full 360-degree camera path for that particular frame and body pose. This task is particularly challenging, as it requires synthesizing photorealistic details of the body, as seen from various camera angles that may not exist in the input video, as well as synthesizing fine details such as cloth folds and facial appearance. Our method optimizes for a volumetric representation of the person in a canonical T-pose, in concert with a motion field that maps the estimated canonical representation to every frame of the video via backward warps. The motion field is decomposed into skeletal rigid and non-rigid motions, produced by deep networks. We show significant performance improvements over prior work, and compelling examples of free-viewpoint renderings from monocular video of moving humans in challenging uncontrolled capture scenarios.

Key facts. Cited by 483 papers in the map (data through 7 Oct 2026). Builds on 45 of its 78 references that are in the map. First posted on arXiv 11 Jan 2022.

Builds on

45 of 78 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  4. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  6. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  8. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  10. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  12. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
33 more
  1. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  2. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  3. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  4. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  5. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  6. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  7. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  8. 2018Video-to-Video Synthesiscited by 388
  9. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  10. 2018Everybody Dance Nowcited by 313
  11. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  12. 2017Pose Guided Person Image Generationcited by 251
  13. 2018Video Based Reconstruction of 3D People Modelscited by 237
  14. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  15. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  16. 2021Neural Articulated Radiance Fieldcited by 196
  17. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  18. 2021ARCH++: Animation-Ready Clothed Human Reconstruction Revisitedcited by 174
  19. 2021H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motioncited by 157
  20. 2021Editable Free-viewpoint Video Using a Layered Neural Representationcited by 148
  21. 2021Real-time Deep Dynamic Characterscited by 133
  22. 2020LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registrationcited by 108
  23. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  24. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  25. 2018Synthesizing Images of Humans in Unseen Posescited by 96
  26. 2018Dense Pose Transfercited by 83
  27. 2021S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modelingcited by 59
  28. 2021SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimizationcited by 59
  29. 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
  30. 2020Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wildcited by 35
  31. 2021Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistencycited by 15
  32. 2021Dance In the Wild: Monocular Human Animation with Neural Dynamic Appearance Synthesiscited by 14
  33. 2019Volumetric Capture of Humans with a Single RGBD Camera via Semi-Parametric Learningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 47; 2023: 157; 2024: 131; 2025: 101; 2026: 47.

483 papers.

Sort:

By area:

  1. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  2. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  3. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  4. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  5. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  6. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  7. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  8. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  9. 2024A Survey on 3D Gaussian Splattingcited by 221
  10. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  11. 2024VastGaussian: Vast 3D Gaussians for Large Scene Reconstructioncited by 209
  12. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 199
38 more
  1. 20233DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splattingcited by 195
  2. 2023Robust Dynamic Radiance Fieldscited by 193
  3. 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 190
  4. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  5. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  6. 2023GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesiscited by 170
  7. 2023GauHuman: Articulated Gaussian Splatting from Monocular Human Videoscited by 168
  8. 20243DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videoscited by 156
  9. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  10. 2023HUGS: Human Gaussian Splatscited by 154
  11. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  12. 2022InstantAvatar: Learning Avatars from Monocular Video in 60 Secondscited by 138
  13. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  14. 2023SparseGS: Sparse View Synthesis using 3D Gaussian Splattingcited by 134
  15. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  16. 2023DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splattingcited by 124
  17. 2023GART: Gaussian Articulated Template Modelscited by 121
  18. 2023Drivable 3D Gaussian Avatarscited by 120
  19. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  20. 2023Human Gaussian Splatting: Real-time Rendering of Animatable Avatarscited by 105
  21. 2023ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Renderingcited by 104
  22. 2022DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenescited by 103
  23. 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
  24. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  25. 20243D Gaussian as a New Era: A Surveycited by 102
  26. 2022Generative Neural Articulated Radiance Fieldscited by 101
  27. 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
  28. 2024Expressive Whole-Body 3D Gaussian Avatarcited by 89
  29. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  30. 2023MonoHuman: Animatable Human Neural Field from Monocular Videocited by 85
  31. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  32. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  33. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
  34. 2022BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fieldscited by 80
  35. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  36. 2023Neural Residual Radiance Fields for Streamably Free-Viewpoint Videoscited by 77
  37. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  38. 2024Fully Explicit Dynamic Gaussian Splattingcited by 75

50 most cited of 483 All 483 (JSON)