mlsanityData to 7 Oct 2026

NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation

arXiv 2203.04802 ↗First posted 9 Mar 2022cs.CVcited by 57

Not on the map. Why?

Abstract (arXiv)

Pose estimation of 3D objects in monocular images is a fundamental and long-standing problem in computer vision. Existing deep learning approaches for 6D pose estimation typically rely on the assumption of availability of 3D object models and 6D pose annotations. However, precise annotation of 6D poses in real data is intricate, time-consuming and not scalable, while synthetic data scales well but lacks realism. To avoid these problems, we present a weakly-supervised reconstruction-based pipeline, named NeRF-Pose, which needs only 2D object segmentation and known relative camera poses during training. Following the first-reconstruct-then-regress idea, we first reconstruct the objects from multiple views in the form of an implicit neural representation. Then, we train a pose regression network to predict pixel-wise 2D-3D correspondences between images and the reconstructed model. At inference, the approach only needs a single image as input. A NeRF-enabled PnP+RANSAC algorithm is used to estimate stable and accurate pose from the predicted correspondences. Experiments on LineMod and LineMod-Occlusion show that the proposed method has state-of-the-art accuracy in comparison to the best 6D pose estimation methods in spite of being trained only with weak labels. Besides, we extend the Homebrewed DB dataset with more real training images to support the weakly supervised task and achieve compelling results on this dataset. The extended dataset and code will be released soon.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 44 of its 70 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

44 of 70 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2017Mask R-CNNcited by 7,259
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  8. 2015Fast R-CNNcited by 2,930
  9. 2018YOLOv3: An Incremental Improvementcited by 2,652
  10. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  11. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  12. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
32 more
  1. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  2. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  3. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  4. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  5. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  6. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 381
  7. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  8. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  9. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  10. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  11. 2017A General and Adaptive Robust Loss Functioncited by 236
  12. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  13. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  14. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  15. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  16. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  17. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  18. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  19. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  20. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  21. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 147
  22. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 144
  23. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  24. 2020HybridPose: 6D Object Pose Estimation under Hybrid Representationscited by 107
  25. 2021SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddingscited by 95
  26. 2019Single-Stage 6D Object Pose Estimationcited by 88
  27. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 88
  28. 2021SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimationcited by 88
  29. 2020Self6D: Self-Supervised Monocular 6D Object Pose Estimationcited by 79
  30. 2018Explaining the Ambiguity of Object Detection and 6D Pose From Visual Datacited by 68
  31. 2019Background Subtraction in Real Applications: Challenges, Current Models and Future Directionscited by 29
  32. 2020I Like to Move It: 6D Pose Estimation as an Action Decision Processcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 16; 2024: 18; 2025: 11; 2026: 5.

57 papers.

Sort:

By area:

  1. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  2. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 114
  3. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  4. 2022Parallel Inversion of Neural Radiance Fields for Robust Pose Estimationcited by 55
  5. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  6. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  7. 2024MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Imagescited by 28
  8. 2023BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fieldscited by 27
  9. 2022TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimationcited by 23
  10. 2023Challenges for Monocular 6D Object Pose Estimation in Roboticscited by 21
  11. 2024Neural Fields in Robotics: A Surveycited by 21
  12. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
38 more
  1. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14
  2. 2023JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fieldscited by 13
  3. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  4. 2024Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Mapscited by 13
  5. 2025Co-op: Correspondence-based Novel Object Pose Estimationcited by 13
  6. 2023SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variationscited by 12
  7. 2023PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matchingcited by 12
  8. 2023iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estimation via Comparing and Matchingcited by 12
  9. 2023Targeted Adversarial Attacks on Generalizable Neural Radiance Fieldscited by 10
  10. 2024Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimationcited by 10
  11. 2024Generic Objects as Pose Probes for Few-shot View Synthesiscited by 9
  12. 2022NeRF-Loc: Transformer-Based Object Localization Within Neural Radiance Fieldscited by 7
  13. 2022Zero NeRF: Registration with Zero Overlapcited by 7
  14. 2025SparSplat: Fast Multi-View Reconstruction with Generalizable 2D Gaussian Splattingcited by 7
  15. 2024Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initializationcited by 5
  16. 2025PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Trackingcited by 5
  17. 2023Factored Neural Representation for Scene Understandingcited by 4
  18. 2024Extreme Two-View Geometry From Object Poses with Diffusion Modelscited by 4
  19. 2024NeRF-Feat: 6D Object Pose Estimation using Feature Renderingcited by 4
  20. 2024SUP-NeRF: A Streamlined Unification of Pose Estimation and NeRF for Monocular 3D Object Reconstructioncited by 3
  21. 2022Surface Normal Clustering for Implicit Representation of Manhattan Scenescited by 2
  22. 2023Open Challenges for Monocular Single-shot 6D Object Pose Estimationcited by 2
  23. 20243D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objectscited by 2
  24. 2024Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondencescited by 2
  25. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single Viewcited by 2
  26. 2022PixTrack: Precise 6DoF Object Pose Tracking using NeRF Templates and Feature-metric Alignmentcited by 1
  27. 2023MonoVisual3DFilter: 3D tomatoes' localisation with monocular cameras using histogram filterscited by 1
  28. 2024Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimationcited by 1
  29. 2024Pose Estimation of Buried Deep-Sea Objects using 3D Vision Deep Learning Modelscited by 1
  30. 2024Generalizable Single-view Object Pose Estimation by Two-side Generating and Matchingcited by 1
  31. 2025Multi-Modal 3D Mesh Reconstruction from Images and Textcited by 1
  32. 2025Category-Level 6D Object Pose Estimation in Agricultural Settings Using a Lattice-Deformation Framework and Diffusion-Augmented Synthetic Datacited by 1
  33. 2023BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fieldscited by 0
  34. 2023Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular Videocited by 0
  35. 2024BVE + EKF: A viewpoint estimator for the estimation of the object's position in the 3D task space using Extended Kalman Filterscited by 0
  36. 2024SABER-6D: Shape Representation Based Implicit Object Pose Estimationcited by 0
  37. 2024GSGTrack: Gaussian Splatting-Guided Object Pose Tracking from RGB Videoscited by 0
  38. 2025Object Pose Estimation through Dexterous Touchcited by 0

50 most cited of 57 All 57 (JSON)