# SpaceSense-Bench: A Large-Scale Multi-Modal Benchmark for Spacecraft Perception and Pose Estimation

arXiv: https://arxiv.org/abs/2603.09320 · HTML: https://mlsanity.com/paper/2603.09320/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 14 of 14)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Satellite Pose Estimation Challenge: Dataset, Competition Design and Results](https://mlsanity.com/paper/1911.02050/) (2019)
- [SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2110.03101/) (2021)
- [Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering](https://mlsanity.com/paper/1907.04298/) (2019)
- [Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models](https://mlsanity.com/paper/2412.18605/) (2024)
- [Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2203.04275/) (2022)
- [A Spacecraft Dataset for Detection, Segmentation and Parts Recognition](https://mlsanity.com/paper/2106.08186/) (2021)
- [Pose Estimation for Non-Cooperative Rendezvous Using Neural Networks](https://mlsanity.com/paper/1906.09868/) (2019)
- [YOLO26: An Analysis of NMS-Free End to End Framework for Real-Time Object Detection](https://mlsanity.com/paper/2601.12882/) (2026)

## Built on by (most cited first, 2 of 2)

- [SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing](https://mlsanity.com/paper/2604.14399/) (2026)
- [GeoDistill-Refine: Silhouette-First Geometry Distillation for Annotation-Free Spacecraft Segmentation](https://mlsanity.com/paper/2608.07405/) (2026)

## Abstract (arXiv)

Autonomous space operations such as on-orbit servicing and active debris removal demand robust part-level semantic understanding and precise relative navigation of target spacecraft, yet collecting large-scale real data in orbit remains impractical due to cost and access constraints. Existing synthetic datasets, moreover, suffer from limited target diversity, single-modality sensing, and incomplete ground-truth annotations. We present SpaceSense-Bench, a large-scale multi-modal benchmark for spacecraft perception encompassing 136~satellite models with approximately 70~GB of data. Each frame provides time-synchronized 1024×1024 RGB images, millimeter-precision depth maps, and 256-beam LiDAR point clouds, together with dense 7-class part-level semantic labels at both the pixel and point level as well as accurate 6-DoF pose ground truth. The dataset is generated through a high-fidelity space simulation built in Unreal Engine~5 and a fully automated pipeline covering data acquisition, multi-stage quality control, and conversion to mainstream formats. We benchmark five representative tasks (object detection, 2D semantic segmentation, RGB--LiDAR fusion-based 3D point cloud segmentation, monocular depth estimation, and orientation estimation) and identify two key findings: (i)~perceiving small-scale components (e.g., thrusters and omni-antennas) and generalizing to entirely unseen spacecraft in a zero-shot setting remain critical bottlenecks for current methods, and (ii)~scaling up the number of training satellites yields substantial performance gains on novel targets, underscoring the value of large-scale, diverse datasets for space perception research. The dataset, code, and toolkit are publicly available at https://github.com/wuaodi/SpaceSense-Bench.