mlsanityData to 14 Oct 2026

OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding

arXiv 2511.12614 ↗First posted 16 Nov 2025cs.CV cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation with a versatile onboarding process. Our pipeline begins with an onboarding stage that generates object representations from either traditional 3D CAD models or, in their absence, by rapidly reconstructing a high-fidelity neural representation (NeRF) from multi-view images. Given a test image, our system first employs the CNOS detector to localize target objects. For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose. The core of OPFormer is a transformer-based architecture that leverages a foundation model for robust feature extraction. It uniquely learns a comprehensive object representation by jointly encoding multiple template views and enriches these features with explicit 3D geometric priors using Normalized Object Coordinate Space (NOCS). A decoder then establishes robust 2D-3D correspondences to determine the final pose. Evaluated on the challenging BOP benchmarks, our integrated system demonstrates a strong balance between accuracy and efficiency, showcasing its practical applicability in both model-based and model-free scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 16 Nov 2025.

Builds on

58 of 71 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  7. 2020GLU Variants Improve Transformercited by 1,804
  8. 2023Vision Transformers Need Registerscited by 871
  9. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  10. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  11. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  12. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
38 more
  1. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  2. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  3. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  4. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  5. 2024Rotary Position Embedding for Vision Transformercited by 236
  6. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  7. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  8. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  9. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  10. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 201
  11. 2020BOP Challenge 2020 on 6D Object Localizationcited by 193
  12. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  13. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 157
  14. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 129
  15. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  16. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 121
  17. 2023SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimationcited by 117
  18. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 116
  19. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 115
  20. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 114
  21. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  22. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  23. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  24. 2021SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddingscited by 95
  25. 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
  26. 2023CNOS: A Strong Baseline for CAD-based Novel Object Segmentationcited by 87
  27. 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 86
  28. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  29. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  30. 2015Recovering 6D Object Pose and Predicting Next-Best-View in the Crowdcited by 79
  31. 2022Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networkscited by 74
  32. 2022NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimationcited by 58
  33. 2023FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation modelscited by 57
  34. 2023SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimationcited by 53
  35. 2023HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructionscited by 48
  36. 2023ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformerscited by 45
  37. 2024Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimationcited by 43
  38. 2023VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representationscited by 41

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinementcited by 0