mlsanityData to 14 Oct 2026

MagicPony: Learning Articulated 3D Animals in the Wild

arXiv 2211.12497 ↗First posted 22 Nov 2022cs.CVcited by 112

Not on the map. Why?

Abstract (arXiv)

We consider the problem of predicting the 3D shape, articulation, viewpoint, texture, and lighting of an articulated animal like a horse given a single test image as input. We present a new method, dubbed MagicPony, that learns this predictor purely from in-the-wild single-view images of the object category, with minimal assumptions about the topology of deformation. At its core is an implicit-explicit representation of articulated shape and appearance, combining the strengths of neural fields and meshes. In order to help the model understand an object's shape and pose, we distil the knowledge captured by an off-the-shelf self-supervised vision transformer and fuse it into the 3D model. To overcome local optima in viewpoint estimation, we further introduce a new viewpoint sampling scheme that comes at no additional training cost. MagicPony outperforms prior work on this challenging task and demonstrates excellent generalisation in reconstructing art, despite the fact that it is only trained on real images.

Key facts. Cited by 112 papers in the map (data through 14 Oct 2026). Builds on 61 of its 81 references that are in the map. First posted on arXiv 22 Nov 2022.

Builds on

61 of 81 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  10. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  11. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  12. 2018Group Normalizationcited by 1,524
38 more
  1. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  2. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  3. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  4. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  6. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  7. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  8. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  9. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  10. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  11. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  12. 2019PointRend: Image Segmentation as Renderingcited by 440
  13. 2017Neural 3D Mesh Renderercited by 437
  14. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  15. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  16. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  17. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  18. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 312
  19. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  20. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  21. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  22. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  23. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  24. 2021Neural Articulated Radiance Fieldcited by 196
  25. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  26. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  27. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  28. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  29. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 116
  30. 2019Neural Cages for Detail-Preserving 3D Deformationscited by 106
  31. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  32. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  33. 2020Articulation-aware Canonical Surface Mappingcited by 81
  34. 2020Continuous Surface Embeddingscited by 80
  35. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
  36. 2021Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifoldcited by 65
  37. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 63
  38. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 28; 2024: 32; 2025: 24; 2026: 28.

112 papers.

Sort:

By area:

  1. 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
  2. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 147
  3. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  4. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 125
  5. 2023GART: Gaussian Articulated Template Modelscited by 121
  6. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 116
  7. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  8. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 74
  9. 2024Learning the 3D Fauna of the Webcited by 59
  10. 2023Reconstructing Animatable Categories from Videoscited by 47
  11. 2023Amodal Ground Truth and Completion in the Wildcited by 45
  12. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
38 more
  1. 2024Score-Guided Diffusion for 3D Human Recoverycited by 39
  2. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 38
  3. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  4. 2023Improving Semantic Correspondence with Viewpoint-Guided Spherical Mapscited by 34
  5. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  6. 2023AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillationcited by 28
  7. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  8. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  9. 2024Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videoscited by 26
  10. 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 25
  11. 2024Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesiscited by 24
  12. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 23
  13. 2025Particulate: Feed-Forward 3D Object Articulationcited by 23
  14. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collectionscited by 22
  15. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  16. 2023CAD: Photorealistic 3D Generation via Adversarial Distillationcited by 18
  17. 2024MagicPose4D: Crafting Articulated Models with Appearance and Motion Controlcited by 18
  18. 2023SAOR: Single-View Articulated Object Reconstructioncited by 16
  19. 2023Template-free Articulated Neural Point Clouds for Reposable View Synthesiscited by 15
  20. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  21. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  22. 2025RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videoscited by 14
  23. 2025It's a (Blind) Match! Towards Vision-Language Correspondence without Parallel Datacited by 14
  24. 2023Cross-domain Compositing with Pretrained Diffusion Modelscited by 13
  25. 2024Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesiscited by 13
  26. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  27. 2023UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fieldscited by 12
  28. 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
  29. 2024AWOL: Analysis WithOut synthesis using Languagecited by 12
  30. 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Videocited by 12
  31. 2024SHIC: Shape-Image Correspondences with no Keypoint Supervisioncited by 12
  32. 2024DressRecon: Freeform 4D Human Reconstruction from Monocular Videocited by 12
  33. 2024Generative Zoocited by 12
  34. 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstructioncited by 11
  35. 2023CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videoscited by 10
  36. 2025MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videoscited by 10
  37. 2024Learning Implicit Representation for Reconstructing Articulated Objectscited by 9
  38. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priorscited by 9

50 most cited of 112 All 112 (JSON)