mlsanityData to 7 Oct 2026

CRISP: Object Pose and Shape Estimation with Test-Time Adaptation

arXiv 2412.01052 ↗First posted 2 Dec 2024cs.CV cs.ROcited by 9

Not on the map. Why?

Abstract (arXiv)

We consider the problem of estimating object pose and shape from an RGB-D image. Our first contribution is to introduce CRISP, a category-agnostic object pose and shape estimation pipeline. The pipeline implements an encoder-decoder model for shape estimation. It uses FiLM-conditioning for implicit shape reconstruction and a DPT-based network for estimating pose-normalized points for pose estimation. As a second contribution, we propose an optimization-based pose and shape corrector that can correct estimation errors caused by a domain gap. Observing that the shape decoder is well behaved in the convex hull of known shapes, we approximate the shape decoder with an active shape model, and show that this reduces the shape correction problem to a constrained linear least squares problem, which can be solved efficiently by an interior point algorithm. Third, we introduce a self-training pipeline to perform self-supervised domain adaptation of CRISP. The self-training is based on a correct-and-certify approach, which leverages the corrector to generate pseudo-labels at test time, and uses them to self-train CRISP. We demonstrate CRISP (and the self-training) on YCBV, SPE3R, and NOCS datasets. CRISP shows high performance on all the datasets. Moreover, our self-training is capable of bridging a large domain gap. Finally, CRISP also shows an ability to generalize to unseen objects. Code and pre-trained models will be available on https://web.mit.edu/sparklab/research/crisp_object_pose_shape/.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 42 of its 52 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

42 of 52 references.

References · 42

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2017Mask R-CNNcited by 7,259
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  8. 2021Vision Transformers for Dense Predictioncited by 1,789
  9. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  12. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
30 more
  1. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  2. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  3. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  4. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  5. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  6. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 198
  7. 2020BOP Challenge 2020 on 6D Object Localizationcited by 192
  8. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  9. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  10. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 155
  11. 2019PRNet: Self-Supervised Learning for Partial-to-Partial Registrationcited by 146
  12. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  13. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  14. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 114
  15. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  16. 20176-DoF Object Pose from Semantic Keypointscited by 89
  17. 2020Self6D: Self-Supervised Monocular 6D Object Pose Estimationcited by 79
  18. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 70
  19. 2024BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objectscited by 60
  20. 2022Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Datasetcited by 59
  21. 2019Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinementcited by 56
  22. 2024Beyond Model Adaptation at Test Time: A Surveycited by 55
  23. 2021Category-Level Metric Scale Object Shape and Pose Estimationcited by 40
  24. 2023TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimationcited by 32
  25. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 18
  26. 2024EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Visioncited by 18
  27. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  28. 2020The State of Industrial Robotics: Emerging Technologies, Challenges, and Key Research Directionscited by 7
  29. 2024Unsupervised Template-assisted Point Cloud Shape Correspondence Networkcited by 7
  30. 2023A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Trainingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 20256D Object Pose Tracking in Internet Videos for Robotic Manipulationcited by 10
  2. 2025Fast Learning of Non-Cooperative Spacecraft 3D Models through Primitive Initializationcited by 5
  3. 2026Picasso: Holistic Scene Reconstruction with Physics-Constrained Samplingcited by 4
  4. 2025Box Pose and Shape Estimation and Domain Adaptation for Large-Scale Warehouse Automationcited by 2
  5. 2025Category-Level Object Shape and Pose Estimation in Less Than a Millisecondcited by 2
  6. 2025H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flowscited by 1
  7. 2026Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perceptioncited by 0
  8. 2026Object Pose and Shape Estimation for Grasping: Does it Work?cited by 0
  9. 2026Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimationcited by 0