AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priors
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
We present AHOY, a method for reconstructing complete, animatable 3D Gaussian avatars from in-the-wild monocular video despite heavy occlusion. Existing methods assume unoccluded input-a fully visible subject, often in a canonical pose-excluding the vast majority of real-world footage where people are routinely occluded by furniture, objects, or other people. Reconstructing from such footage poses fundamental challenges: large body regions may never be observed, and multi-view supervision per pose is unavailable. We address these challenges with four contributions: (i) a hallucination-as-supervision pipeline that uses identity-finetuned diffusion models to generate dense supervision for previously unobserved body regions; (ii) a two-stage canonical-to-pose-dependent architecture that bootstraps from sparse observations to full pose-dependent Gaussian maps; (iii) a map-pose/LBS-pose decoupling that absorbs multi-view inconsistencies from the generated data; (iv) a head/body split supervision strategy that preserves facial identity. We evaluate on YouTube videos and on multi-view capture data with significant occlusion and demonstrate state-of-the-art reconstruction quality. We also demonstrate that the resulting avatars are robust enough to be animated with novel poses and composited into 3DGS scenes captured using cell-phone video. Our project page is available at https://miraymen.github.io/ahoy/
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 140 of its 155 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
140 of 155 references.
Closest ancestors · 11
Same direction, newest first
- 2025MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction31 Jul
- 2025Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatars21 Jul
- 2025ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs23 Jun
- 2025MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention11 Mar
- 2025Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs7 Mar
- 2025GAS: Generative Avatar Synthesis from a Single Image10 Feb
- 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models16 Dec
- 2024MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement26 Aug
- 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humans16 Aug
- 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models3 Apr
- 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
Other references · 129
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
38 more
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2017End-to-end Recovery of Human Shape and Posecited by 885
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
- 2020First Order Motion Model for Image Animationcited by 704
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
- 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
- 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
- 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
- 2021Neural Fields in Visual Computing and Beyondcited by 540
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
- 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 388
- 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
- 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
- 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
- 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
- 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
- 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
- 2018Video Based Reconstruction of 3D People Modelscited by 238
- 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
50 most cited of 129 All 129 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion
- 2025SVAD: From Single Image to 3D Avatar via Synthetic Data Generation with Video Diffusion and Data Augmentation
- 2024SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wild
- 20243D2-Actor: Learning Pose-Conditioned 3D-Aware Denoiser for Realistic Gaussian Avatar Modeling
- 2026GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
- 2025Controlling Avatar Diffusion with Learnable Gaussian Embedding
- 2025Towards Efficient 3D Gaussian Human Avatar Compression: A Prior-Guided Framework
- 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation