# A Self-Supervised Framework for Space Object Behaviour Characterisation

arXiv: https://arxiv.org/abs/2504.06176 · HTML: https://mlsanity.com/paper/2504.06176/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables](https://mlsanity.com/paper/2402.19072/) (2024)
- [Trustchain -- Trustworthy Decentralised Public Key Infrastructure for Digital Credentials](https://mlsanity.com/paper/2305.08533/) (2023)

## Abstract (arXiv)

Foundation Models, which leverage large neural networks pre-trained on unlabelled data before fine-tuning for specific tasks, are increasingly being applied to specialised domains. Recent examples include ClimaX for climate and Clay for satellite Earth observation, but a Foundation Model for Space Object Behavioural Analysis has not yet been developed. As orbital populations grow, automated methods for characterising space object behaviour are crucial for space safety. Here, we present a self-supervised framework for space object behavioural analysis, representing a first step towards a Foundation Model for SOBA. The backbone is a Perceiver-Variational Autoencoder (VAE) architecture, pre-trained with self-supervised reconstruction and masked reconstruction on ~227,000 light curves from the MMT-9 observatory. The VAE enables anomaly detection, motion prediction, and synthetic light curve generation. We fine-tuned the model using two independent light curve simulators (CASSANDRA and GRIAL), with CAD models of boxwing, Sentinel-3, SMOS, and Starlink platforms. Our pre-trained model achieved a reconstruction mean squared error of 0.0012, identifying potentially anomalous light curves through reconstruction difficulty. After fine-tuning, the model scored 85% and 82% accuracy, with 0.92 and 0.95 ROC AUC scores in anomaly detection and motion mode prediction (e.g., sun-pointing, spin, tumbling). Analysis of high-confidence predictions on real data revealed distinct patterns including characteristic object profiles and satellite glinting. Our work demonstrates how self-supervised learning can simultaneously enable anomaly detection, motion prediction, and synthetic data generation from rich pre-trained representations, supporting space safety and sustainability through automated monitoring and simulation.