# Cross-Modal RGB-D Fusion Transformer for 6D Pose Estimation of Non-Cooperative Spacecraft with Stereo-Derived Depth

arXiv: https://arxiv.org/abs/2605.08592 · HTML: https://mlsanity.com/paper/2605.08592/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 18 of 18)

- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [Rotate to Attend: Convolutional Triplet Attention Module](https://mlsanity.com/paper/2010.03045/) (2020)
- [SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications](https://mlsanity.com/paper/2303.15446/) (2023)
- [SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2110.03101/) (2021)
- [Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering](https://mlsanity.com/paper/1907.04298/) (2019)
- [Wide-Depth-Range 6D Object Pose Estimation in Space](https://mlsanity.com/paper/2104.00337/) (2021)
- [Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2203.04275/) (2022)
- [Leveraging Equivariant Features for Absolute Pose Regression](https://mlsanity.com/paper/2204.02163/) (2022)
- [Depth-based 6DoF Object Pose Estimation using Swin Transformer](https://mlsanity.com/paper/2303.02133/) (2023)
- [TransPose: 6D Object Pose Estimation with Geometry-Aware Transformer](https://mlsanity.com/paper/2310.16279/) (2023)

## Abstract (arXiv)

On-orbit servicing and active debris removal involving non-cooperative spacecraft require reliable pose estimation to supply accurate position and orientation data for autonomous visual navigation. Learning-based monocular methods have seen widespread adoption in spacecraft pose estimation, yet they suffer from an intrinsic depth ambiguity problem and tend to fail under the harsh illumination conditions routinely encountered in orbit. Active depth sensors could in principle address the geometric ambiguity, but their power and mass requirements make them poorly suited to most spacecraft platforms. This work addresses these issues through a passive stereo vision framework for six-degree-of-freedom (6-DOF) pose estimation of non-cooperative spacecraft. A binocular stereo matching network called TSCA-Stereo is developed to cope with weak-texture surfaces, specular highlights, and severe lighting variations typical of space imagery. A cross-modal fusion Transformer is introduced to combine RGB appearance information with stereo depth features in an adaptive manner, supporting reliable pose recovery. A synthetic binocular multimodal dataset is also built for the experiments, covering stereo disparity maps and 6-DOF pose annotations across a range of illumination scenarios, attitude configurations, and noise levels. Experimental results show that TSCA-Stereo outperforms the baseline across every evaluated metric on this space-specific dataset. The full pose estimation pipeline achieves a mean translation error of 0.0419 m and a mean orientation error of 0.8632deg under varied imaging conditions, confirming that the passive stereo approach is both effective and resilient when operating under the demanding visual conditions of the space environment.