mlsanityData to 14 Oct 2026

Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimation

arXiv 2308.05438 ↗First posted 10 Aug 2023cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

One critical challenge in 6D object pose estimation from a single RGBD image is efficient integration of two different modalities, i.e., color and depth. In this work, we tackle this problem by a novel Deep Fusion Transformer~(DFTr) block that can aggregate cross-modality features for improving pose estimation. Unlike existing fusion methods, the proposed DFTr can better model cross-modality semantic correlation by leveraging their semantic similarity, such that globally enhanced features from different modalities can be better integrated for improved information extraction. Moreover, to further improve robustness and efficiency, we introduce a novel weighted vector-wise voting algorithm that employs a non-iterative global optimization strategy for precise 3D keypoint localization while achieving near real-time inference. Extensive experiments show the effectiveness and strong generalization capability of our proposed 3D keypoint voting algorithm. Results on four widely used benchmarks also demonstrate that our method outperforms the state-of-the-art methods by large margins.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 49 of its 68 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

49 of 68 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2016Pyramid Scene Parsing Networkcited by 2,479
  5. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  6. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  7. 2017Focal Loss for Dense Object Detectioncited by 752
  8. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  9. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  10. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  11. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  12. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
37 more
  1. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  2. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  3. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  4. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  5. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  6. 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
  7. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  8. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  9. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  10. 2019PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimationcited by 204
  11. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  12. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  13. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  14. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 124
  15. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 116
  16. 2020HybridPose: 6D Object Pose Estimation under Hybrid Representationscited by 107
  17. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  18. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 101
  19. 2018In Defense of Classical Image Processing: Fast Depth Completion on the CPUcited by 96
  20. 2019LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimationcited by 89
  21. 2019Single-Stage 6D Object Pose Estimationcited by 88
  22. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  23. 2021Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networkscited by 69
  24. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  25. 2018A Unified Framework for Multi-View Multi-Class Object Pose Estimationcited by 49
  26. 2020MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusioncited by 46
  27. 2020G2L-Net: Global to Local Network for Real-time 6D Pose Estimation with Embedding Vector Featurescited by 35
  28. 2017Going Further with Point Pair Featurescited by 28
  29. 2021Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Votingcited by 25
  30. 2022Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimationcited by 22
  31. 2022Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Pickingcited by 20
  32. 2022StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCScited by 18
  33. 2021PR-GCN: A Deep Graph Convolutional Network with Point Refinement for 6D Pose Estimationcited by 17
  34. 2021Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimationcited by 14
  35. 2022DGECN: A Depth-Guided Edge Convolutional Network for End-to-End 6D Pose Estimationcited by 12
  36. 2019MaskedFusion: Mask-based 6D Object Pose Estimationcited by 9
  37. 2018Vision-based Pose Estimation for Augmented Reality : A Comparison Studycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 6; 2026: 1.

16 papers.

Sort:

  1. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 40
  2. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  3. 2024MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Imagescited by 28
  4. 2023HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimationcited by 17
  5. 2025Scalable Unseen Objects 6-DoF Absolute Pose Estimation with Robotic Integrationcited by 11
  6. 2024GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinementcited by 8
  7. 2025GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimationcited by 6
  8. 2024RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Imagescited by 5
  9. 2023Pseudo-keypoint RKHS Learning for Self-supervised 6DoF Pose Estimationcited by 4
  10. 2025Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypescited by 4
  11. 2023Learning Better Keypoints for Multi-Object 6DoF Pose Estimationcited by 3
  12. 2024KGpose: Keypoint-Graph Driven End-to-End Multi-Object 6D Pose Estimation via Point-Wise Pose Votingcited by 0
4 more
  1. 2025DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarioscited by 0
  2. 2025DLTPose: 6DoF Pose Estimation From Accurate Dense Surface Point Estimatescited by 0
  3. 2025RCGNet: RGB-based Category-Level 6D Object Pose Estimation with Geometric Guidancecited by 0
  4. 2026G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulationcited by 0