mlsanityData to 7 Oct 2026

PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

arXiv 2209.07589 ↗First posted 15 Sep 2022cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Estimating the relative pose of a new object without prior knowledge is a hard problem, while it is an ability very much needed in robotics and Augmented Reality. We present a method for tracking the 6D motion of objects in RGB video sequences when neither the training images nor the 3D geometry of the objects are available. In contrast to previous works, our method can therefore consider unknown objects in open world instantly, without requiring any prior information or a specific training phase. We consider two architectures, one based on two frames, and the other relying on a Transformer Encoder, which can exploit an arbitrary number of past frames. We train our architectures using only synthetic renderings with domain randomization. Our results on challenging datasets are on par with previous works that require much more information (training images of the target objects, 3D models, and/or depth data). Our source code is available at https://github.com/nv-nguyen/pizza

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

43 of 59 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  7. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  8. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  9. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  10. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  11. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  12. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
31 more
  1. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  2. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  3. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  4. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  5. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  6. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  7. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  8. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 155
  9. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  10. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 120
  11. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  12. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  13. 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
  14. 2019PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Trackingcited by 91
  15. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 88
  16. 2018DeepTAM: Deep Tracking and Mappingcited by 87
  17. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  18. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  19. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  20. 2015Learning Descriptors for Object Recognition and 3D Pose Estimationcited by 60
  21. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  22. 2019Multi-path Learning for Object Pose Estimation Across Domainscited by 52
  23. 2015A Dataset for Improved RGBD-based Object Detection and Pose Estimation for Warehouse Pick-and-Placecited by 35
  24. 2020I Like to Move It: 6D Pose Estimation as an Action Decision Processcited by 22
  25. 20203D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddingscited by 22
  26. 2019Location Field Descriptors: Single Image 3D Model Retrieval in the Wildcited by 15
  27. 2020Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Imagescited by 14
  28. 2017Deep 6-DOF Trackingcited by 13
  29. 2021Learning to Better Segment Objects from Unseen Classes with Unlabeled Videoscited by 9
  30. 20211st Place Solution for the UVO Challenge on Image-based Open-World Segmentation 2021cited by 8
  31. 2019Few-Shot Viewpoint Estimationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 2; 2025: 3; 2026: 2.

15 papers.

Sort:

  1. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  2. 2023RelPose++: Recovering 6D Poses from Sparse-view Observationscited by 87
  3. 2023CNOS: A Strong Baseline for CAD-based Novel Object Segmentationcited by 86
  4. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  5. 2023NOPE: Novel Object Pose Estimation from a Single Imagecited by 38
  6. 2023Open-vocabulary object 6D pose estimationcited by 29
  7. 2025Scalable Unseen Objects 6-DoF Absolute Pose Estimation with Robotic Integrationcited by 11
  8. 2025GoTrack: Generic 6DoF Object Pose Refinement and Trackingcited by 10
  9. 2023KeyMatchNet: Zero-Shot Pose Estimation in 3D Point Clouds by Generalized Keypoint Matchingcited by 7
  10. 2022Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensingcited by 5
  11. 2023Tracking by 3D Model Estimation of Unknown Objects in Videoscited by 5
  12. 2026Event6D: Event-based Novel Object 6D Pose Trackingcited by 2
3 more
  1. 2025SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Referencecited by 1
  2. 2024OmniPose6D: Towards Short-Term Object Pose Tracking in Dynamic Scenes from Monocular RGBcited by 0
  3. 2026Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perceptioncited by 0