mlsanityData to 14 Oct 2026

CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects

arXiv 2303.15782 ↗First posted 28 Mar 2023cs.CV cs.ROcited by 64

Not on the map. Why?

Abstract (arXiv)

We present CARTO, a novel approach for reconstructing multiple articulated objects from a single stereo RGB observation. We use implicit object-centric representations and learn a single geometry and articulation decoder for multiple object categories. Despite training on multiple categories, our decoder achieves a comparable reconstruction accuracy to methods that train bespoke decoders separately for each category. Combined with our stereo image encoder we infer the 3D shape, 6D pose, size, joint type, and the joint state of multiple unknown objects in a single forward pass. Our method achieves a 20.4% absolute improvement in mAP 3D IOU50 for novel instances when compared to a two-stage pipeline. Inference time is fast and can run on a NVIDIA TITAN XP GPU at 1 HZ for eight or less objects present. While only trained on simulated data, CARTO transfers to real-world object instances. Code and evaluation data is available at: http://carto.cs.uni-freiburg.de

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 40 of its 46 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

40 of 46 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Mask R-CNNcited by 7,277
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  5. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  6. 2020Convolutional Occupancy Networkscited by 761
  7. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  8. 2021Neural Fields in Visual Computing and Beyondcited by 540
  9. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  10. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  11. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 180
  12. 2019Category-Level Articulated Object Pose Estimationcited by 159
28 more
  1. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 156
  2. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 116
  3. 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 114
  4. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 108
  5. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  6. 2021CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Cloudscited by 84
  7. 2022AKB-48: A Real-World Articulated Object Knowledge Basecited by 82
  8. 2019DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Datacited by 77
  9. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 72
  10. 2020ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theorycited by 70
  11. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 70
  12. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  13. 2018Robust Watertight Manifold Surface Generation Method for ShapeNet Modelscited by 59
  14. 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 57
  15. 2022CLA-NeRF: Category-Level Articulated Neural Radiance Fieldcited by 56
  16. 2022Self-supervised Neural Articulated Shape and Appearance Modelscited by 51
  17. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  18. 2014A Probabilistic Framework for Learning Kinematic Models of Articulated Objectscited by 40
  19. 2018The RBO Dataset of Articulated Objects and Interactionscited by 34
  20. 2022SPAMs: Structured Implicit Parametric Modelscited by 29
  21. 2022CATRE: Iterative Point Clouds Alignment for Category-level Object Pose Refinementcited by 29
  22. 2022Category-Independent Articulated Object Tracking with Factor Graphscited by 24
  23. 2021SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereocited by 18
  24. 2021A Learned Stereo Depth System for Robotic Manipulation in Homescited by 14
  25. 2022Unsupervised Kinematic Motion Detection for Part-segmented 3D Shape Collectionscited by 13
  26. 2021OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrievalcited by 12
  27. 2019Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Roboticscited by 10
  28. 2022ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 16; 2025: 22; 2026: 18.

64 papers.

Sort:

By area:

  1. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 118
  2. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 111
  3. 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 69
  4. 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 69
  5. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
  6. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  7. 2023CAGE: Controllable Articulation GEnerationcited by 52
  8. 2024REACTO: Reconstructing Articulated Objects from a Single Videocited by 45
  9. 2024MeshArt: Generating Articulated Meshes with Structure-Guided Transformerscited by 41
  10. 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 35
  11. 2025ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splattingcited by 35
  12. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
38 more
  1. 2024Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesiscited by 24
  2. 2024ArtFormer: Controllable Generation of Diverse 3D Articulated Objectscited by 22
  3. 2025FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusioncited by 22
  4. 2024Neural Fields in Robotics: A Surveycited by 21
  5. 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 21
  6. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 19
  7. 2025Articulated Object Estimation in the Wildcited by 19
  8. 2024LEIA: Latent View-invariant Embeddings for Implicit 3D Articulationcited by 16
  9. 2025VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Videocited by 15
  10. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  11. 2023CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimationcited by 13
  12. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  13. 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Graspingcited by 13
  14. 2025SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splattingcited by 13
  15. 20233D Reconstruction of Objects in Hands without Real World 3D Supervisioncited by 12
  16. 2025AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learningcited by 12
  17. 2025LARM: A Large Articulated-Object Reconstruction Modelcited by 12
  18. 2025ScrewSplat: An End-to-End Method for Articulated Object Recognitioncited by 9
  19. 2025ART: Articulated Reconstruction Transformercited by 9
  20. 2025Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understandingcited by 8
  21. 2026FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videoscited by 8
  22. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  23. 2026Articulation in Motion: Prior-free Part Mobility Analysis for Articulated Objects By Dynamic-Static Disentanglementcited by 6
  24. 2024SM3: Self-Supervised Multi-task Modeling with Multi-view 2D Images for Articulated Objectscited by 5
  25. 2024PAPR in Motion: Seamless Point-level 3D Scene Interpolationcited by 5
  26. 2024Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Imagescited by 5
  27. 2024AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registrationcited by 5
  28. 2025Part Segmentation and Motion Estimation for Articulated Objects with Dynamic 3D Gaussianscited by 5
  29. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  30. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Modelscited by 4
  31. 2026Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraintscited by 3
  32. 2026FreeArtGS: Articulated Gaussian Splatting Under Free-moving Scenariocited by 3
  33. 2025sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Onlycited by 2
  34. 2026MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twinscited by 2
  35. 2024CenterArt: Joint Shape Reconstruction and 6-DoF Grasp Estimation of Articulated Objectscited by 1
  36. 2025Part2GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splattingcited by 1
  37. 2025PAOLI: Pose-free Articulated Object Learning from Sparse-view Imagescited by 1
  38. 2026ArtPro: Self-Supervised Articulated Object Reconstruction with Adaptive Integration of Mobility Proposalscited by 1

50 most cited of 64 All 64 (JSON)