# ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

arXiv: https://arxiv.org/abs/2306.04619 · HTML: https://mlsanity.com/paper/2306.04619/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 43 of its 46 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [DreamBooth3D: Subject-Driven Text-to-3D Generation](https://mlsanity.com/paper/2303.13508/) (2023)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [Text-To-4D Dynamic Scene Generation](https://mlsanity.com/paper/2301.11280/) (2023)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"](https://mlsanity.com/paper/1908.07201/) (2019)

## Built on by (most cited first, 22 of 22)

- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion](https://mlsanity.com/paper/2403.12008/) (2024)
- [State of the Art on Diffusion Models for Visual Computing](https://mlsanity.com/paper/2310.07204/) (2023)
- [Learning the 3D Fauna of the Web](https://mlsanity.com/paper/2401.02400/) (2024)
- [NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations](https://mlsanity.com/paper/2306.09109/) (2023)
- [MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusion](https://mlsanity.com/paper/2310.14729/) (2023)
- [AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation](https://mlsanity.com/paper/2312.03795/) (2023)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos](https://mlsanity.com/paper/2312.13604/) (2023)
- [BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors](https://mlsanity.com/paper/2403.11427/) (2024)
- [Survey on Modeling of Human-made Articulated Objects](https://mlsanity.com/paper/2403.14937/) (2024)
- [DreaMo: Articulated 3D Reconstruction From A Single Casual Video](https://mlsanity.com/paper/2312.02617/) (2023)
- [AWOL: Analysis WithOut synthesis using Language](https://mlsanity.com/paper/2404.03042/) (2024)
- [DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction](https://mlsanity.com/paper/2412.04464/) (2024)
- [Kinematify: Open-Vocabulary Synthesis of High-DoF Articulated Objects](https://mlsanity.com/paper/2511.01294/) (2025)
- [Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images](https://mlsanity.com/paper/2410.03438/) (2024)
- [Scene-Conditional 3D Object Stylization and Composition](https://mlsanity.com/paper/2312.12419/) (2023)
- [Model-based Metric 3D Shape and Motion Reconstruction of Wild Bottlenose Dolphins in Drone-Shot Videos](https://mlsanity.com/paper/2504.15782/) (2025)
- [DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recovery](https://mlsanity.com/paper/2510.24117/) (2025)
- [Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animals](https://mlsanity.com/paper/2508.16062/) (2025)
- [YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals](https://mlsanity.com/paper/2406.16273/) (2024)
- [CORGI: Consistency-Aware 3D Dog Reconstruction from a Single Image in the Wild](https://mlsanity.com/paper/2607.00321/) (2026)

## Abstract (arXiv)

Estimating 3D articulated shapes like animal bodies from monocular images is inherently challenging due to the ambiguities of camera viewpoint, pose, texture, lighting, etc. We propose ARTIC3D, a self-supervised framework to reconstruct per-instance 3D shapes from a sparse image collection in-the-wild. Specifically, ARTIC3D is built upon a skeleton-based surface representation and is further guided by 2D diffusion priors from Stable Diffusion. First, we enhance the input images with occlusions/truncation via 2D diffusion to obtain cleaner mask estimates and semantic features. Second, we perform diffusion-guided 3D optimization to estimate shape and texture that are of high-fidelity and faithful to input images. We also propose a novel technique to calculate more stable image-level gradients via diffusion models compared to existing alternatives. Finally, we produce realistic animations by fine-tuning the rendered shape and texture under rigid part transformations. Extensive evaluations on multiple existing datasets as well as newly introduced noisy web image collections with occlusions and truncation demonstrate that ARTIC3D outputs are more robust to noisy images, higher quality in terms of shape and texture details, and more realistic when animated. Project page: https://chhankyao.github.io/artic3d/