# PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modeling

arXiv: https://arxiv.org/abs/2403.16080 · HTML: https://mlsanity.com/paper/2403.16080/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 57 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Builds on (most cited first, 30 of 37)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction](https://mlsanity.com/paper/2212.05231/) (2022)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [TAVA: Template-free Animatable Volumetric Actors](https://mlsanity.com/paper/2206.08929/) (2022)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [Efficient Neural Radiance Fields for Interactive Free-viewpoint Video](https://mlsanity.com/paper/2112.01517/) (2021)
- [Editable Free-viewpoint Video Using a Layered Neural Representation](https://mlsanity.com/paper/2104.14786/) (2021)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Real-time Deep Dynamic Characters](https://mlsanity.com/paper/2105.01794/) (2021)
- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs](https://mlsanity.com/paper/2112.02789/) (2021)
- [Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control](https://mlsanity.com/paper/2106.02019/) (2021)

## Built on by (most cited first, 18 of 18)

- [A Benchmark for Gaussian Splatting Compression and Quality Assessment Study](https://mlsanity.com/paper/2407.14197/) (2024)
- [A Hierarchical Compression Technique for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2411.06976/) (2024)
- [HybridGS: High-Efficiency Gaussian Splatting Data Compression using Dual-Channel Sparse Representation and Point Cloud Encoder](https://mlsanity.com/paper/2505.01938/) (2025)
- [GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data](https://mlsanity.com/paper/2411.18624/) (2024)
- [Free-viewpoint Human Animation with Pose-correlated Reference Selection](https://mlsanity.com/paper/2412.17290/) (2024)
- [3DGS-VBench: A Comprehensive Video Quality Evaluation Benchmark for 3DGS Compression](https://mlsanity.com/paper/2508.07038/) (2025)
- [WildAvatar: Learning In-the-wild 3D Avatars from the Web](https://mlsanity.com/paper/2407.02165/) (2024)
- [ViDiC: Video Difference Captioning](https://mlsanity.com/paper/2512.03405/) (2025)
- [GaussianTrimmer: Online Trimming Boundaries for 3DGS Segmentation](https://mlsanity.com/paper/2601.12683/) (2026)
- [SynPlay: Large-Scale Synthetic Human Data with Real-World Diversity for Aerial-View Perception](https://mlsanity.com/paper/2408.11814/) (2024)
- [ViVo: A Dataset for Volumetric Video Reconstruction and Compression](https://mlsanity.com/paper/2506.00558/) (2025)
- [Online Segment 3D Gaussians via Launching Virtual Drones](https://mlsanity.com/paper/2607.01628/) (2026)
- [Sliding Window Informative Canonical Correlation Analysis](https://mlsanity.com/paper/2507.17921/) (2025)
- [DCVC-MV: Deep Contextual Multiview Video Compression with Efficient Inter-View Prediction](https://mlsanity.com/paper/2509.03922/) (2025)
- [MV-Fashion: Towards Enabling Virtual Try-On and Size Estimation with Multi-View Paired Data](https://mlsanity.com/paper/2603.08147/) (2026)
- [Does it matter which Gaussians you pick in 4D Gaussian streaming?](https://mlsanity.com/paper/2603.17227/) (2026)
- [PointSplat: Compact Gaussian Splatting via Human-Centric Prediction](https://mlsanity.com/paper/2606.32036/) (2026)
- [Gaussian Splatting for Real-Time Scene Reconstruction: Methods, Systems, and Trade-offs](https://mlsanity.com/paper/2610.17513/) (2026)

## Abstract (arXiv)

High-quality human reconstruction and photo-realistic rendering of a dynamic scene is a long-standing problem in computer vision and graphics. Despite considerable efforts invested in developing various capture systems and reconstruction algorithms, recent advancements still struggle with loose or oversized clothing and overly complex poses. In part, this is due to the challenges of acquiring high-quality human datasets. To facilitate the development of these fields, in this paper, we present PKU-DyMVHumans, a versatile human-centric dataset for high-fidelity reconstruction and rendering of dynamic human scenarios from dense multi-view videos. It comprises 8.2 million frames captured by more than 56 synchronized cameras across diverse scenarios. These sequences comprise 32 human subjects across 45 different scenarios, each with a high-detailed appearance and realistic human motion. Inspired by recent advancements in neural radiance field (NeRF)-based scene representations, we carefully set up an off-the-shelf framework that is easy to provide those state-of-the-art NeRF-based implementations and benchmark on PKU-DyMVHumans dataset. It is paving the way for various applications like fine-grained foreground/background decomposition, high-quality human reconstruction and photo-realistic novel view synthesis of a dynamic scene. Extensive studies are performed on the benchmark, demonstrating new observations and challenges that emerge from using such high-fidelity dynamic data.