# A Markov Decision Process Framework for Early Maneuver Decisions in Satellite Collision Avoidance

arXiv: https://arxiv.org/abs/2508.05876 · HTML: https://mlsanity.com/paper/2508.05876/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Spacecraft Collision Avoidance Challenge: design and results of a machine learning competition](https://mlsanity.com/paper/2008.03069/) (2020)
- [Spacecraft Collision Risk Assessment with Probabilistic Programming](https://mlsanity.com/paper/2012.10260/) (2020)
- [Analytical framework for space debris collision avoidance maneuver design](https://mlsanity.com/paper/2012.12651/) (2020)
- [Collision Avoidance Maneuver Optimization with a Multiple-Impulse Convex Formulation](https://mlsanity.com/paper/2101.07403/) (2021)
- [Conjunction Data Messages for Space Collision Behave as a Poisson Process](https://mlsanity.com/paper/2303.15074/) (2023)
- [Recursive Polynomial Method for Fast Collision Avoidance Maneuver Design](https://mlsanity.com/paper/2406.01949/) (2024)

## Built on by (most cited first, 1 of 1)

- [Chance-Constrained Belief-Space Maneuver Planning for Autonomous Collision Avoidance Under Uncertainty](https://mlsanity.com/paper/2609.13428/) (2026)

## Abstract (arXiv)

We develop a Markov decision process (MDP) framework to autonomously make guidance decisions for satellite collision avoidance maneuver (CAM) and a reinforcement learning policy gradient (RL-PG) algorithm to enable direct optimization of guidance policy using historic CAM data. In addition to maintaining acceptable collision risks, this approach seeks to minimize the average propellant consumption of CAMs by making early maneuver decisions. We model CAM as a continuous state, discrete action and finite horizon MDP, where the critical decision is determining when to initiate the maneuver. The MDP models decision rewards using analytical models of collision risk, propellant consumption, and transit orbit geometry. By deciding to maneuver earlier than conventional methods, the Markov policy effectively favors CAMs that achieve comparable rates of collision risk reduction while consuming less propellant. Using historical data of tracked conjunction events, we verify this framework and conduct an extensive parameter-sensitivity study. When evaluated on synthetic conjunction events, the trained policy consumes significantly less propellant overall and per maneuver in comparison to a conventional cut-off policy that initiates maneuvers 24 hours before the time of closest approach (TCA). On historical conjunction events, the trained policy consumes more propellant overall but consumes less propellant per maneuver. For both historical and synthetic conjunction events, the trained policy is slightly more conservative in identifying conjunctions events that warrant CAMs in comparison to cutoff policies.