# Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos

arXiv: https://arxiv.org/abs/2403.17103 · HTML: https://mlsanity.com/paper/2403.17103/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 58 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [Animatable Neural Radiance Fields from Monocular RGB Videos](https://mlsanity.com/paper/2106.13629/) (2021)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding](https://mlsanity.com/paper/1711.06475/) (2017)
- [ViTPose++: Vision Transformer for Generic Body Pose Estimation](https://mlsanity.com/paper/2212.04246/) (2022)
- [Category-Specific Object Reconstruction from a Single Image](https://mlsanity.com/paper/1411.6069/) (2014)

## Built on by (most cited first, 26 of 26)

- [AniMer: Animal Pose and Shape Estimation Using Family Aware Transformer](https://mlsanity.com/paper/2412.00837/) (2024)
- [Generative Zoo](https://mlsanity.com/paper/2412.08101/) (2024)
- [AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformer](https://mlsanity.com/paper/2508.00298/) (2025)
- [4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos](https://mlsanity.com/paper/2507.10437/) (2025)
- [DIMO: Diverse 3D Motion Generation for Arbitrary Objects](https://mlsanity.com/paper/2511.07409/) (2025)
- [LIM: Large Interpolator Model for Dynamic Reconstruction](https://mlsanity.com/paper/2503.22537/) (2025)
- [ReVision: Refining Video Diffusion with Explicit 3D Motion Modeling](https://mlsanity.com/paper/2504.21855/) (2025)
- [Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field](https://mlsanity.com/paper/2505.10049/) (2025)
- [DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recovery](https://mlsanity.com/paper/2510.24117/) (2025)
- [NeuralFur: Animal Fur Reconstruction From Multi-View Images](https://mlsanity.com/paper/2601.12481/) (2026)
- [Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance](https://mlsanity.com/paper/2505.18342/) (2025)
- [4DEquine: Disentangling Motion and Appearance for 4D Equine Reconstruction from Monocular Video](https://mlsanity.com/paper/2603.10125/) (2026)
- [RatBodyFormer: Rat Body Surface from Keypoints](https://mlsanity.com/paper/2412.09599/) (2024)
- [MoReMouse: Monocular Reconstruction of Laboratory Mouse](https://mlsanity.com/paper/2507.04258/) (2025)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequences](https://mlsanity.com/paper/2602.12740/) (2026)
- [BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations](https://mlsanity.com/paper/2602.19874/) (2026)
- [SMAL-pets: SMAL Based Avatars of Pets from Single Image](https://mlsanity.com/paper/2603.17131/) (2026)
- [PRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptation](https://mlsanity.com/paper/2606.02366/) (2026)
- [InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation](https://mlsanity.com/paper/2607.10287/) (2026)
- [Kirin: Animal Motion Generation from In-the-Wild Video](https://mlsanity.com/paper/2609.01823/) (2026)
- [DogWeave: High-Fidelity 3D Canine Reconstruction from a Single Image via Normal Fusion and Conditional Inpainting](https://mlsanity.com/paper/2603.07441/) (2026)
- [Progressive Pose-Guided 4D Animal Reconstruction from Monocular Video](https://mlsanity.com/paper/2607.00157/) (2026)
- [CORGI: Consistency-Aware 3D Dog Reconstruction from a Single Image in the Wild](https://mlsanity.com/paper/2607.00321/) (2026)
- [AnimalLift: Reconstructing Animatable 3D Animals from a Single Image by Learning Canonical Shape, Texture, and Fur Maps](https://mlsanity.com/paper/2609.09513/) (2026)
- [ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animals](https://mlsanity.com/paper/2609.37986/) (2026)

## Abstract (arXiv)

We present a method to build animatable dog avatars from monocular videos. This is challenging as animals display a range of (unpredictable) non-rigid movements and have a variety of appearance details (e.g., fur, spots, tails). We develop an approach that links the video frames via a 4D solution that jointly solves for animal's pose variation, and its appearance (in a canonical pose). To this end, we significantly improve the quality of template-based shape fitting by endowing the SMAL parametric model with Continuous Surface Embeddings, which brings image-to-mesh reprojection constaints that are denser, and thus stronger, than the previously used sparse semantic keypoint correspondences. To model appearance, we propose an implicit duplex-mesh texture that is defined in the canonical pose, but can be deformed using SMAL pose coefficients and later rendered to enforce a photometric compatibility with the input video frames. On the challenging CoP3D and APTv2 datasets, we demonstrate superior results (both in terms of pose estimates and predicted appearance) to existing template-free (RAC) and template-based approaches (BARC, BITE).