# SpY: A Context-Based Approach to Spacecraft Component Detection

arXiv: https://arxiv.org/abs/2406.18709 · HTML: https://mlsanity.com/paper/2406.18709/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2110.03101/) (2021)
- [Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft](https://mlsanity.com/paper/1909.00392/) (2019)
- [A Spacecraft Dataset for Detection, Segmentation and Parts Recognition](https://mlsanity.com/paper/2106.08186/) (2021)
- [Real-Time, Flight-Ready, Non-Cooperative Spacecraft Pose Estimation Using Monocular Imagery](https://mlsanity.com/paper/2101.09553/) (2021)
- [ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation](https://mlsanity.com/paper/2108.10282/) (2021)
- [LSPnet: A 2D Localization-oriented Spacecraft Pose Estimation Neural Network](https://mlsanity.com/paper/2104.09248/) (2021)
- [Performance Study of YOLOv5 and Faster R-CNN for Autonomous Navigation around Non-Cooperative Targets](https://mlsanity.com/paper/2301.09056/) (2023)
- [Characterizing Satellite Geometry via Accelerated 3D Gaussian Splatting](https://mlsanity.com/paper/2401.02588/) (2024)
- [Autonomous Rendezvous with Non-cooperative Target Objects with Swarm Chasers and Observers](https://mlsanity.com/paper/2301.09059/) (2023)
- [SpaceYOLO: A Human-Inspired Model for Real-time, On-board Spacecraft Feature Detection](https://mlsanity.com/paper/2302.00824/) (2023)
- [Using Convolutional Neural Networks for Relative Pose Estimation of a Non-Cooperative Spacecraft with Thermal Infrared Imagery](https://mlsanity.com/paper/2105.13789/) (2021)
- [Resource-constrained FPGA Design for Satellite Component Feature Extraction](https://mlsanity.com/paper/2301.09055/) (2023)
- [Taking a PEEK into YOLOv5 for Satellite Component Recognition via Entropy-based Visual Explanations](https://mlsanity.com/paper/2311.01703/) (2023)

## Abstract (arXiv)

This paper focuses on autonomously characterizing components such as solar panels, body panels, antennas, and thrusters of an unknown resident space object (RSO) using camera feed to aid autonomous on-orbit servicing (OOS) and active debris removal. Significant research has been conducted in this area using convolutional neural networks (CNNs). While CNNs are powerful at learning patterns and performing object detection, they struggle with missed detections and misclassifications in environments different from the training data, making them unreliable for safety in high-stakes missions like OOS. Additionally, failures exhibited by CNNs are often easily rectifiable by humans using commonsense reasoning and contextual knowledge. Embedding such reasoning in an object detector could improve detection accuracy. To validate this hypothesis, this paper presents an end-to-end object detector called SpaceYOLOv2 (SpY), which leverages the generalizability of CNNs while incorporating contextual knowledge using traditional computer vision techniques. SpY consists of two main components: a shape detector and the SpaceYOLO classifier (SYC). The shape detector uses CNNs to detect primitive shapes of RSOs and SYC associates these shapes with contextual knowledge, such as color and texture, to classify them as spacecraft components or "unknown" if the detected shape is uncertain. SpY's modular architecture allows customizable usage of contextual knowledge to improve detection performance, or SYC as a secondary fail-safe classifier with an existing spacecraft component detector. Performance evaluations on hardware-in-the-loop images of a mock-up spacecraft demonstrate that SpY is accurate and an ensemble of SpY with YOLOv5 trained for satellite component detection improved the performance by 23.4% in recall, demonstrating enhanced safety for vision-based navigation tasks.