# Multi-Phase Spacecraft Trajectory Optimization via Transformer-Based Reinforcement Learning

arXiv: https://arxiv.org/abs/2511.11402 · HTML: https://mlsanity.com/paper/2511.11402/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- First posted on arXiv 14 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Built on by (most cited first, 4 of 4)

- [Tiny Recursive Control: Iterative Reasoning for Efficient Optimal Control](https://mlsanity.com/paper/2512.16824/) (2025)
- [Autonomous Reasoning for Spacecraft Control: A Large Language Model Framework with Group Relative Policy Optimization](https://mlsanity.com/paper/2601.04334/) (2026)
- [Agentic AutoResearch forSpace Autonomy: An Auditable, LLM-Driven Research Agent for Aerospace Control Problems](https://mlsanity.com/paper/2606.20394/) (2026)
- [Spacecraft Rendezvous Trajectory Generation with Modular Constraints via Diffusion Model Composition](https://mlsanity.com/paper/2610.05642/) (2026)

## Abstract (arXiv)

Autonomous spacecraft control for mission phases such as launch, ascent, stage separation, and orbit insertion remains a critical challenge due to the need for adaptive policies that generalize across dynamically distinct regimes. While reinforcement learning (RL) has shown promise in individual astrodynamics tasks, existing approaches often require separate policies for distinct mission phases, limiting adaptability and increasing operational complexity. This work introduces a transformer-based RL framework that unifies multi-phase trajectory optimization through a single policy architecture, leveraging the transformer's inherent capacity to model extended temporal contexts. Building on proximal policy optimization (PPO), our framework replaces conventional recurrent networks with a transformer encoder-decoder structure, enabling the agent to maintain coherent memory across mission phases spanning seconds to minutes during critical operations. By integrating a Gated Transformer-XL (GTrXL) architecture, the framework eliminates manual phase transitions while maintaining stability in control decisions. We validate our approach progressively: first demonstrating near-optimal performance on single-phase benchmarks (double integrator and Van der Pol oscillator), then extending to multiphase waypoint navigation variants, and finally tackling a complex multiphase rocket ascent problem that includes atmospheric flight, stage separation, and vacuum operations. Results demonstrate that the transformer-based framework not only matches analytical solutions in simple cases but also effectively learns coherent control policies across dynamically distinct regimes, establishing a foundation for scalable autonomous mission planning that reduces reliance on phase-specific controllers while maintaining compatibility with safety-critical verification protocols.