mlsanityData to 7 Oct 2026

MVTOP: Multi-View Transformer-based Object Pose-Estimation

arXiv 2508.03243 ↗First posted 5 Aug 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present MVTOP, a novel transformer-based method for multi-view rigid object pose estimation. Through an early fusion of the view-specific features, our method can resolve pose ambiguities that would be impossible to solve with a single view or with a post-processing of single-view poses. MVTOP models the multi-view geometry via lines of sight that emanate from the respective camera centers. While the method assumes the camera interior and relative orientations are known for a particular scene, they can vary for each inference. This makes the method versatile. The use of the lines of sight enables MVTOP to correctly predict the correct pose with the merged multi-view information. To show the model's capabilities, we provide a synthetic data set that can only be solved with such holistic multi-view approaches since the poses in the dataset cannot be solved with just one view. Our method outperforms single-view and all existing multi-view approaches on our dataset and achieves competitive results on the YCB-V dataset. To the best of our knowledge, no holistic multi-view method exists that can resolve such pose ambiguities reliably. Our model is end-to-end trainable and does not require any additional data, e.g., depth.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 62 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

38 of 62 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2017Mask R-CNNcited by 7,259
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  6. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  7. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  8. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  9. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  10. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  11. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  12. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
26 more
  1. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  2. 2020BOP Challenge 2020 on 6D Object Localizationcited by 192
  3. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  4. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  5. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 147
  6. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  7. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  8. 2021SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimationcited by 88
  9. 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 85
  10. 2015Recovering 6D Object Pose and Predicting Next-Best-View in the Crowdcited by 79
  11. 2023Dense Distinct Query for End-to-End Object Detectioncited by 76
  12. 2016Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challengecited by 70
  13. 2021Direct Multi-view Multi-person 3D Pose Estimationcited by 64
  14. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  15. 2018A Unified Framework for Multi-View Multi-Class Object Pose Estimationcited by 49
  16. 2021CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulationcited by 49
  17. 2023HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructionscited by 47
  18. 2017Real-time Perception meets Reactive Motion Generationcited by 30
  19. 2021ROBI: A Multi-View Dataset for Reflective Objects in Robotic Bin-Pickingcited by 22
  20. 2025BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose Estimationcited by 18
  21. 2021T6D-Direct: Transformers for Multi-Object 6D Pose Direct Regressioncited by 16
  22. 2018SilhoNet: An RGB Method for 6D Object Pose Estimationcited by 14
  23. 2022PoET: Pose Estimation Transformer for Single-View, Multi-Object 6D Pose Estimationcited by 13
  24. 2019CullNet: Calibrated and Pose Aware Confidence Scores for Object Pose Estimationcited by 9
  25. 2022MV6D: Multi-View 6D Pose Estimation on RGB-D Frames Using a Deep Point-wise Voting Networkcited by 9
  26. 2018Dealing with Ambiguity in Robotic Grasping via Multiple Predictionscited by 4

Built on by

Not cited yet.