# Transformer-Based Reinforcement Learning for Autonomous Orbital Collision Avoidance in Partially Observable Environments

arXiv: https://arxiv.org/abs/2602.06088 · HTML: https://mlsanity.com/paper/2602.06088/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [TransDreamer: Reinforcement Learning with Transformer World Models](https://mlsanity.com/paper/2202.09481/) (2022)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [Fast Trajectory Optimization via Successive Convexification for Spacecraft Rendezvous with Integer Constraints](https://mlsanity.com/paper/1906.04857/) (2019)
- [A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2109.09889/) (2021)
- [Social Robot Navigation through Constrained Optimization: a Comparative Study of Uncertainty-based Objectives and Constraints](https://mlsanity.com/paper/2305.02859/) (2023)
- [Recursive Polynomial Method for Fast Collision Avoidance Maneuver Design](https://mlsanity.com/paper/2406.01949/) (2024)
- [Risk-Sensitive Orbital Debris Collision Avoidance using Distributionally Robust Chance Constraints](https://mlsanity.com/paper/2412.17358/) (2024)
- [Collision Avoidance Maneuvers Optimization in the Presence of Multiple Encounters](https://mlsanity.com/paper/2406.03654/) (2024)
- [Chance-constrained Model Predictive Control for Near Rectilinear Halo Orbit spacecraft rendezvous](https://mlsanity.com/paper/2501.10437/) (2025)
- [CAMmary: A Review of Spacecraft Collision Avoidance Manoeuvre Design Methods](https://mlsanity.com/paper/2503.22555/) (2025)
- [Successive Convexification for Passively-Safe Spacecraft Rendezvous on Near Rectilinear Halo Orbit](https://mlsanity.com/paper/2505.17251/) (2025)

## Abstract (arXiv)

We introduce a Transformer-based Reinforcement Learning framework for autonomous orbital collision avoidance that explicitly models the effects of partial observability and imperfect monitoring in space operations. The framework combines a configurable encounter simulator, a distance-dependent observation model, and a sequential state estimator to represent uncertainty in relative motion. A central contribution of this work is the use of transformer-based Partially Observable Markov Decision Process (POMDP) architecture, which leverage long-range temporal attention to interpret noisy and intermittent observations more effectively than traditional architectures. This integration provides a foundation for training collision avoidance agents that can operate more reliably under imperfect monitoring environments.