mlsanityData to 7 Oct 2026

ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization

arXiv 2207.13691 ↗First posted 27 Jul 2022cs.CV cs.LG cs.ROcited by 70

Not on the map. Why?

Abstract (arXiv)

Our method studies the complex task of object-centric 3D understanding from a single RGB-D observation. As it is an ill-posed problem, existing methods suffer from low performance for both 3D shape and 6D pose and size estimation in complex multi-object scenarios with occlusions. We present ShAPO, a method for joint multi-object detection, 3D textured reconstruction, 6D object pose and size estimation. Key to ShAPO is a single-shot pipeline to regress shape, appearance and pose latent codes along with the masks of each object instance, which is then further refined in a sparse-to-dense fashion. A novel disentangled shape and appearance database of priors is first learned to embed objects in their respective shape and appearance space. We also propose a novel, octree-based differentiable optimization step, allowing us to further improve object shape, pose and appearance simultaneously under the learned latent space, in an analysis-by-synthesis fashion. Our novel joint implicit textured object representation allows us to accurately identify and reconstruct novel unseen objects without having access to their 3D meshes. Through extensive experiments, we show that our method, trained on simulated indoor scenes, accurately regresses the shape, appearance and pose of novel objects in the real-world with minimal fine-tuning. Our method significantly out-performs all baselines on the NOCS dataset with an 8% absolute improvement in mAP for 6D pose estimation. Project page: https://zubair-irshad.github.io/projects/ShAPO.html

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Builds on 57 of its 64 references that are in the map. First posted on arXiv 27 Jul 2022.

Builds on

57 of 64 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Mask R-CNNcited by 7,259
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  10. 2019Objects as Pointscited by 1,172
  11. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
38 more
  1. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  2. 2018Panoptic Segmentationcited by 796
  3. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  4. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  5. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  6. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  7. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  8. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  9. 2019Panoptic Feature Pyramid Networkscited by 644
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  12. 2017Neural 3D Mesh Renderercited by 437
  13. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  14. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  15. 2018PCN: Point Completion Networkcited by 382
  16. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  17. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  18. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  19. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  20. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  21. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  22. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  23. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  24. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  25. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  26. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 147
  27. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  28. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  29. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
  30. 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
  31. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  32. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  33. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  34. 2019Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priorscited by 74
  35. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 69
  36. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  37. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  38. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 22; 2024: 20; 2025: 17; 2026: 5.

70 papers.

Sort:

By area:

  1. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  2. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  3. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  4. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
  5. 2023HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimationcited by 47
  6. 2023HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructionscited by 47
  7. 2023ObjectSDF++: Improved Object-Compositional Neural Implicit Surfacescited by 38
  8. 2023TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimationcited by 32
  9. 2023ZeroShape: Regression-based Zero-shot Shape Reconstructioncited by 30
  10. 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimationcited by 24
  11. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  12. 2024Neural Fields in Robotics: A Surveycited by 21
38 more
  1. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  2. 2022Generative Category-Level Shape and Pose Estimation with Semantic Primitivescited by 19
  3. 2023RICO: Regularizing the Unobservable for Indoor Compositional Reconstructioncited by 19
  4. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  5. 2022StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCScited by 18
  6. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 18
  7. 2024NeRFs in Robotics: A Surveycited by 16
  8. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
  9. 2023Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environmentscited by 15
  10. 2023CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimationcited by 13
  11. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  12. 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Graspingcited by 13
  13. 2025SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splattingcited by 13
  14. 2023SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variationscited by 12
  15. 20233D Reconstruction of Objects in Hands without Real World 3D Supervisioncited by 12
  16. 2024Omni6D: Large-Vocabulary 3D Object Dataset for Category-Level 6D Object Pose Estimationcited by 11
  17. 2023Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflectioncited by 10
  18. 20256D Object Pose Tracking in Internet Videos for Robotic Manipulationcited by 10
  19. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  20. 2024Zero-Shot Multi-Object Scene Completioncited by 9
  21. 2024CRISP: Object Pose and Shape Estimation with Test-Time Adaptationcited by 9
  22. 2024GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinementcited by 8
  23. 2024OP-Align: Object-level and Part-level Alignment for Self-supervised Category-level Articulated Object Pose Estimationcited by 8
  24. 2022CPPF++: Uncertainty-Aware Sim2Real Object Pose Estimation by Vote Aggregationcited by 6
  25. 2022ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapescited by 6
  26. 2023Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representationscited by 5
  27. 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Datasetcited by 5
  28. 2024Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Imagescited by 5
  29. 2025GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimationcited by 5
  30. 2024Extreme Two-View Geometry From Object Poses with Diffusion Modelscited by 4
  31. 2022A mixed-reality dataset for category-level 6D pose and size estimation of hand-occluded containerscited by 3
  32. 2023Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videoscited by 3
  33. 2024Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Trackercited by 3
  34. 2025Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypescited by 3
  35. 2023RGB-D-Based Categorical Object Pose and Shape Estimation: Methods, Datasets, and Evaluationcited by 2
  36. 2023Real-time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Predictioncited by 2
  37. 2024ReFiNe: Recursive Field Networks for Cross-modal Multi-scene Representationcited by 2
  38. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single Viewcited by 2

50 most cited of 70 All 70 (JSON)