mlsanityData to 14 Oct 2026

Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects

arXiv 2112.11347 ↗First posted 21 Dec 2021cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

Rendering articulated objects while controlling their poses is critical to applications such as virtual reality or animation for movies. Manipulating the pose of an object, however, requires the understanding of its underlying structure, that is, its joints and how they interact with each other. Unfortunately, assuming the structure to be known, as existing methods do, precludes the ability to work on new object categories. We propose to learn both the appearance and the structure of previously unseen articulated objects by observing them move from multiple views, with no joints annotation supervision, or information about the structure. We observe that 3D points that are static relative to one another should belong to the same part, and that adjacent parts that move relative to each other must be connected by a joint. To leverage this insight, we model the object parts in 3D as ellipsoids, which allows us to identify joints. We combine this explicit representation with an implicit one that compensates for the approximation introduced. We show that our method works for different structures, from quadrupeds, to single-arm robots, to humans.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 61 of its 79 references that are in the map. First posted on arXiv 21 Dec 2021.

Builds on

61 of 79 references.

References · 61

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  3. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  5. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  6. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  7. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  8. 2017End-to-end Recovery of Human Shape and Posecited by 885
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  10. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  11. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
38 more
  1. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  2. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  3. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  4. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  5. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  6. 2018Everybody Dance Nowcited by 315
  7. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 307
  8. 2017Integral Human Pose Regressioncited by 256
  9. 2017Pose Guided Person Image Generationcited by 251
  10. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  11. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  12. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  13. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
  14. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  15. 2021Neural Articulated Radiance Fieldcited by 196
  16. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  17. 2019Learnable Triangulation of Human Posecited by 161
  18. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  19. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  20. 2019NASA: Neural Articulated Shape Approximationcited by 149
  21. 2020BlendMask: Top-Down Meets Bottom-Up for Instance Segmentationcited by 145
  22. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  23. 2018A Variational U-Net for Conditional Appearance and Shape Generationcited by 124
  24. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  25. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  26. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  27. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  28. 2018Extreme View Synthesiscited by 94
  29. 2021Neural Trajectory Fields for Dynamic Novel View Synthesiscited by 86
  30. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  31. 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
  32. 2018Learning Monocular 3D Human Pose Estimation from Multi-view Imagescited by 70
  33. 2019Unsupervised Part-Based Disentangling of Object Shape and Appearancecited by 67
  34. 2020Neural Deformation Graphs for Globally-consistent Non-rigid Reconstructioncited by 67
  35. 2020Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wildcited by 66
  36. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  37. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 61
  38. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 10; 2023: 12; 2024: 9; 2025: 12; 2026: 7.

50 papers.

Sort:

By area:

  1. 2022Deforming Radiance Fields with Cagescited by 78
  2. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
  3. 2022Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representationscited by 65
  4. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  5. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
  6. 2022NeuPhysics: Editable Neural Geometry and Physics from Monocular Videoscited by 50
  7. 2023Reconstructing Animatable Categories from Videoscited by 47
  8. 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
  9. 2022CASA: Category-agnostic Skeletal Animal Reconstructioncited by 35
  10. 2023Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Cloudscited by 34
  11. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  12. 2023AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillationcited by 28
38 more
  1. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  2. 2025HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representationcited by 22
  3. 2022AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypointscited by 17
  4. 2023Template-free Articulated Neural Point Clouds for Reposable View Synthesiscited by 15
  5. 2025VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Videocited by 15
  6. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  7. 2023Zero-shot Pose Transfer for Unrigged Stylized 3D Characterscited by 13
  8. 2024Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesiscited by 13
  9. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  10. 2023MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Fieldcited by 12
  11. 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Videocited by 12
  12. 2022Inferring Articulated Rigid Body Dynamics from RGBD Videocited by 10
  13. 2023CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videoscited by 10
  14. 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labelscited by 9
  15. 2024Learning Implicit Representation for Reconstructing Articulated Objectscited by 9
  16. 2022Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challengescited by 8
  17. 2025Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understandingcited by 8
  18. 2023BlendFields: Few-Shot Example-Driven Facial Modelingcited by 6
  19. 2022One-Shot Neural Fields for 3D Object Understandingcited by 5
  20. 2024SM3: Self-Supervised Multi-task Modeling with Multi-view 2D Images for Articulated Objectscited by 5
  21. 2024AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registrationcited by 5
  22. 2025Part Segmentation and Motion Estimation for Articulated Objects with Dynamic 3D Gaussianscited by 5
  23. 2025Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generationcited by 5
  24. 2022Structure from Action: Learning Interactions for Articulated Object 3D Structure Discoverycited by 4
  25. 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
  26. 2022Unsupervised 3D Keypoint Discovery with Multi-View Geometrycited by 2
  27. 2025Recovering Dynamic 3D Sketches from Videoscited by 2
  28. 2025sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Onlycited by 2
  29. 2025ArticFlow: Generative Simulation of Articulated Mechanismscited by 1
  30. 2026SV-GS: Sparse View 4D Reconstruction with Skeleton-Driven Gaussian Splattingcited by 1
  31. 2026Articulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Videocited by 1
  32. 20233D Pose Nowcasting: Forecast the Future to Improve the Presentcited by 0
  33. 2024Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videoscited by 0
  34. 2025Learning High-Fidelity Robot Self-Model with Articulated 3D Gaussian Splattingcited by 0
  35. 2026GenMatter: Perceiving Physical Objects with Generative Matter Modelscited by 0
  36. 2026StructureGS: Structure-aware Gaussian Splatting for Articulated Object Reconstructioncited by 0
  37. 2026Track, Articulate, Act: Generating Articulation from Casual Human Videoscited by 0
  38. 2026FAMOS: Feed-Forward 3D Articulation Modeling from Sparse Observationscited by 0