WildAvatar: Learning In-the-wild 3D Avatars from the Web
Not on the map. Why?
Abstract (arXiv)
Existing research on avatar creation is typically limited to laboratory datasets, which require high costs against scalability and exhibit insufficient representation of the real world. On the other hand, the web abounds with off-the-shelf real-world human videos, but these videos vary in quality and require accurate annotations for avatar creation. To this end, we propose an automatic annotating pipeline with filtering protocols to curate these humans from the web. Our pipeline surpasses state-of-the-art methods on the EMDB benchmark, and the filtering protocols boost verification metrics on web videos. We then curate WildAvatar, a web-scale in-the-wild human avatar creation dataset extracted from YouTube, with 10000+ different human subjects and scenes. WildAvatar is at least 10× richer than previous datasets for 3D human avatar creation and closer to the real world. To explore its potential, we demonstrate the quality and generalizability of avatar creation methods on WildAvatar. We will publicly release our code, data source links and annotations to push forward 3D human avatar creation and other related fields for real-world applications.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 69 of its 106 references that are in the map. First posted on arXiv 2 Jul 2024.
Builds on
69 of 106 references.
References · 69
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023Segment Anythingcited by 8,158
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2017End-to-end Recovery of Human Shape and Posecited by 883
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
- 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
38 more
- 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
- 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
- 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
- 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
- 2021Advances in Neural Renderingcited by 365
- 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
- 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
- 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
- 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
- 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
- 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
- 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
- 2018Video Based Reconstruction of 3D People Modelscited by 237
- 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 237
- 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
- 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
- 2024GaussianPro: 3D Gaussian Splatting with Progressive Propagationcited by 199
- 2019DeepHuman: 3D Human Reconstruction from a Single Imagecited by 182
- 2020Real-Time High-Resolution Background Mattingcited by 172
- 2023GauHuman: Articulated Gaussian Splatting from Monocular Human Videoscited by 168
- 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
- 2023HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motioncited by 161
- 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
- 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
- 2021Efficient Neural Radiance Fields for Interactive Free-viewpoint Videocited by 150
- 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
- 2021Real-time Deep Dynamic Characterscited by 133
- 2022Structured Local Radiance Fields for Human Avatar Modelingcited by 116
- 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 115
- 2020Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstructioncited by 112
- 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
- 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
- 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 95
- 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 89
- 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
- 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
50 most cited of 69 All 69 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.