mlsanityData to 7 Oct 2026

Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation

arXiv 2205.11047 ↗First posted 23 May 2022cs.CV cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

We propose a single-stage, category-level 6-DoF pose estimation algorithm that simultaneously detects and tracks instances of objects within a known category. Our method takes as input the previous and current frame from a monocular RGB video, as well as predictions from the previous frame, to predict the bounding cuboid and 6-DoF pose (up to scale). Internally, a deep network predicts distributions over object keypoints (vertices of the bounding cuboid) in image coordinates, after which a novel probabilistic filtering process integrates across estimates before computing the final pose using PnP. Our framework allows the system to take previous uncertainties into consideration when predicting the current frame, resulting in predictions that are more accurate and stable than single frame methods. Extensive experiments show that our method outperforms existing approaches on the challenging Objectron benchmark of annotated object videos. We also demonstrate the usability of our work in an augmented reality setting.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

32 of 40 references.

References · 32

  1. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  2. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  3. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  4. 2020Tracking Objects as Pointscited by 483
  5. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  6. 2019Tracking without bells and whistlescited by 382
  7. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  8. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  9. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  10. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  11. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  12. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
20 more
  1. 2017Detect to Track and Track to Detectcited by 131
  2. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 123
  3. 2020A Review and Comparative Study on Probabilistic Object Detection in Autonomous Drivingcited by 115
  4. 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
  5. 2019PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Trackingcited by 91
  6. 2020se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domainscited by 87
  7. 2021CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Cloudscited by 83
  8. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  9. 2016Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challengecited by 70
  10. 2019BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectorscited by 68
  11. 2021Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Imagecited by 42
  12. 2019Uncertainty Estimation in One-Stage Object Detectioncited by 36
  13. 2019Learning an Uncertainty-Aware Object Detector for Autonomous Drivingcited by 34
  14. 2020Probabilistic Oriented Object Detection in Automotive Radarcited by 27
  15. 2016Depth-Based Object Tracking Using a Robust Gaussian Filtercited by 22
  16. 2015Probabilistic Object Tracking using a Range Cameracited by 21
  17. 2018Integrated Object Detection and Tracking with Tracklet-Conditioned Detectioncited by 17
  18. 2020MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervisioncited by 16
  19. 2020Towards Better Performance and More Explainable Uncertainty for 3D Object Detection of Autonomous Vehiclescited by 9
  20. 2018Synthetically Trained Neural Networks for Learning Human-Readable Plans from Real-World Demonstrationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 3; 2025: 5; 2026: 3.

19 papers.

Sort:

  1. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  2. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  3. 2023HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructionscited by 47
  4. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  5. 2023iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estimation via Comparing and Matchingcited by 12
  6. 2023PACE: A Large-Scale Dataset with Pose Annotations in Cluttered Environmentscited by 11
  7. 2023Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensorscited by 10
  8. 2025GoTrack: Generic 6DoF Object Pose Refinement and Trackingcited by 10
  9. 2022Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensingcited by 5
  10. 2025DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objectscited by 5
  11. 2023Neural Textured Deformable Meshes for Robust Analysis-by-Synthesiscited by 4
  12. 2024Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Trackercited by 3
7 more
  1. 2026Event6D: Event-based Novel Object 6D Pose Trackingcited by 2
  2. 2025Uncertainty Quantification for Visual Object Pose Estimation: S-Lemma Ellipsoidal Boundscited by 1
  3. 2026Event-based Motion & Appearance Fusion for 6D Object Pose Trackingcited by 1
  4. 2024OmniPose6D: Towards Short-Term Object Pose Tracking in Dynamic Scenes from Monocular RGBcited by 0
  5. 2025Exploring Category-level Articulated Object Pose Tracking on SE(3) Manifoldscited by 0
  6. 2025Light Field Based 6DoF Tracking of Previously Unobserved Objectscited by 0
  7. 2026Detecting Pose Estimation Failures via Keypoint Self-Consistencycited by 0