mlsanityData to 7 Oct 2026

Creatures great and SMAL: Recovering the shape and motion of animals from video

arXiv 1811.05804 ↗First posted 14 Nov 2018cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

We present a system to recover the 3D shape and motion of a wide variety of quadrupeds from video. The system comprises a machine learning front-end which predicts candidate 2D joint positions, a discrete optimization which finds kinematically plausible joint correspondences, and an energy minimization stage which fits a detailed 3D model to the image. In order to overcome the limited availability of motion capture training data from animals, and the difficulty of generating realistic synthetic training images, the system is designed to work on silhouette data. The joint candidate predictor is trained on synthetically generated silhouette images, and at test time, deep learning methods or standard video segmentation tools are used to extract silhouettes from real data. The system is tested on animal videos from several species, and shows accurate reconstructions of 3D shape and pose.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 11; 2023: 17; 2024: 16; 2025: 15; 2026: 11.

84 papers.

Sort:

By area:

  1. 2023CoTracker: It is Better to Track Togethercited by 516
  2. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  3. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  4. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  5. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  6. 2021Seeing biodiversity: perspectives in machine learning for wildlife conservationcited by 129
  7. 2023GART: Gaussian Articulated Template Modelscited by 121
  8. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  9. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  10. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  11. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  12. 2021OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Associationcited by 55
38 more
  1. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  2. 2022BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Informationcited by 51
  3. 2021From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimationcited by 48
  4. 2023Reconstructing Animatable Categories from Videoscited by 46
  5. 2023Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shapecited by 43
  6. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  7. 2022CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshescited by 41
  8. 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 38
  9. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  10. 2021AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wildcited by 29
  11. 2024Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videoscited by 26
  12. 2024AniMer: Animal Pose and Shape Estimation Using Family Aware Transformercited by 20
  13. 2021hSMAL: Detailed Horse Shape and Pose Reconstruction for Motion Pattern Recognitioncited by 18
  14. 20233D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capturecited by 18
  15. 2023OmniMotionGPT: Animal Motion Generation with Limited Datacited by 18
  16. 2024MagicPose4D: Crafting Articulated Models with Appearance and Motion Controlcited by 18
  17. 2021Coarse-to-fine Animal Pose and Shape Estimationcited by 16
  18. 2021Discovering Relationships between Object Categories via Universal Canonical Mapscited by 13
  19. 20233D-MuPPET: 3D Multi-Pigeon Pose Estimation and Trackingcited by 12
  20. 2024Generative Zoocited by 12
  21. 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Videocited by 11
  22. 2022Self-supervised AutoFlowcited by 10
  23. 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstructioncited by 10
  24. 2021SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimationcited by 9
  25. 2024Learning Implicit Representation for Reconstructing Articulated Objectscited by 9
  26. 2022Artemis: Articulated Neural Pets with Appearance and Motion synthesiscited by 8
  27. 2022Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challengescited by 8
  28. 2025Web-Scale Collection of Video Data for 4D Animal Reconstructioncited by 8
  29. 2024Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Imagescited by 7
  30. 2024Reconstructing Animals and the Wildcited by 7
  31. 2025AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformercited by 7
  32. 2022Adjusting for Bias with Procedural Datacited by 6
  33. 2023KBody: Towards general, robust, and aligned monocular whole-body estimationcited by 6
  34. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  35. 2021DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimensioncited by 5
  36. 2022Aligning Silhouette Topology for Self-Adaptive 3D Human Pose Recoverycited by 5
  37. 2025Surface-Aware Distilled 3D Semantic Featurescited by 5
  38. 2025LIM: Large Interpolator Model for Dynamic Reconstructioncited by 5

50 most cited of 84 All 84 (JSON)