mlsanityData to 7 Oct 2026

Real-time volumetric rendering of dynamic humans

arXiv 2303.11898 ↗First posted 21 Mar 2023cs.CV cs.GRcited by 5

Not on the map. Why?

Abstract (arXiv)

We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits. Our method can reconstruct a dynamic human in less than 3h using a single GPU, compared to recent state-of-the-art alternatives that take up to 72h. These speedups are obtained by using a lightweight deformation model solely based on linear blend skinning, and an efficient factorized volumetric representation for modeling the shape and color of the person in canonical pose. Moreover, we propose a novel local ray marching rendering which, by exploiting standard GPU hardware and without any baking or conversion of the radiance field, allows visualizing the neural human on a mobile VR device at 40 frames per second with minimal loss of visual quality. Our experimental evaluation shows superior or competitive results with state-of-the art methods while obtaining large training speedup, using a simple model, and achieving real-time rendering.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 37 of its 46 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

37 of 46 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  5. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  11. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  12. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
25 more
  1. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  2. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  3. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  4. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  5. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  6. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  7. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  8. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  9. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  10. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  11. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  12. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  13. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  14. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  15. 2020Free View Synthesiscited by 231
  16. 2017Submanifold Sparse Convolutional Networkscited by 223
  17. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  18. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  19. 2021ARCH++: Animation-Ready Clothed Human Reconstruction Revisitedcited by 174
  20. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  21. 2020Stable View Synthesiscited by 165
  22. 2019NASA: Neural Articulated Shape Approximationcited by 149
  23. 2021HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputscited by 112
  24. 2020Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wildcited by 35
  25. 2021Free-Viewpoint RGB-D Human Performance Capture and Renderingcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 2.

5 papers.

  1. 2023CoTracker: It is Better to Track Togethercited by 516
  2. 2024GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Meshcited by 51
  3. 2024Unsupervised Skin Feature Tracking with Deep Neural Networkscited by 0
  4. 2025Unsupervised 2D-3D lifting of non-rigid objects using local constraintscited by 0
  5. 2025Link to the Past: Temporal Propagation for Fast 3D Human Reconstruction from Monocular Videocited by 0