# Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks

arXiv: https://arxiv.org/abs/2507.19535 · HTML: https://mlsanity.com/paper/2507.19535/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 27 of its 59 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?](https://mlsanity.com/paper/2204.05618/) (2022)
- [Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning](https://mlsanity.com/paper/2407.15007/) (2024)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations](https://mlsanity.com/paper/1907.05317/) (2019)
- [Real-time optimal control via Deep Neural Networks: study on landing problems](https://mlsanity.com/paper/1610.08668/) (2016)
- [End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight](https://mlsanity.com/paper/2311.16948/) (2023)
- [A Survey on Artificial Intelligence Trends in Spacecraft Guidance Dynamics and Control](https://mlsanity.com/paper/1812.02948/) (2018)
- [Real-Time Optimal Control for Irregular Asteroid Landings Using Deep Neural Networks](https://mlsanity.com/paper/1901.02210/) (2019)
- [End-to-end Neural Network Based Quadcopter control](https://mlsanity.com/paper/2304.13460/) (2023)
- [Neural representation of a time optimal, constant acceleration rendezvous](https://mlsanity.com/paper/2203.15490/) (2022)
- [Revisiting high-order Taylor methods for astrodynamics and celestial mechanics](https://mlsanity.com/paper/2105.00800/) (2021)
- [Adaptive Generalized ZEM-ZEV Feedback Guidance for Planetary Landing via a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2003.02182/) (2020)
- [Guidance and Control Networks with Periodic Activation Functions](https://mlsanity.com/paper/2405.18084/) (2024)
- [Reliable event detection for Taylor methods in astrodynamics](https://mlsanity.com/paper/2204.09948/) (2022)
- [Optimality Principles in Spacecraft Neural Guidance and Control](https://mlsanity.com/paper/2305.13078/) (2023)
- [Frequency and Generalisation of Periodic Activation Functions in Reinforcement Learning](https://mlsanity.com/paper/2407.06756/) (2024)
- [Six Degree-of-Freedom Body-Fixed Hovering over Unmapped Asteroids via LIDAR Altimetry and Reinforcement Meta-Learning](https://mlsanity.com/paper/1911.08553/) (2019)
- [Distilling Privileged Information for Dubins Traveling Salesman Problems with Neighborhoods](https://mlsanity.com/paper/2404.16721/) (2024)
- [Certifying Guidance & Control Networks: Uncertainty Propagation to an Event Manifold](https://mlsanity.com/paper/2410.03729/) (2024)
- [High-order expansion of Neural Ordinary Differential Equations flows](https://mlsanity.com/paper/2504.08769/) (2025)

## Abstract (arXiv)

Guidance & control networks (G&CNETs) provide a promising alternative to on-board guidance and control (G&C) architectures for spacecraft, offering a differentiable, end-to-end representation of the guidance and control architecture. When training G&CNETs, two predominant paradigms emerge: behavioural cloning (BC), which mimics optimal trajectories, and reinforcement learning (RL), which learns optimal behaviour through trials and errors. Although both approaches have been adopted in G&CNET related literature, direct comparisons are notably absent. To address this, we conduct a systematic evaluation of BC and RL specifically for training G&CNETs on continuous-thrust spacecraft trajectory optimisation tasks. We introduce a novel RL training framework tailored to G&CNETs, incorporating decoupled action and control frequencies alongside reward redistribution strategies to stabilise training and to provide a fair comparison. Our results show that BC-trained G&CNETs excel at closely replicating expert policy behaviour, and thus the optimal control structure of a deterministic environment, but can be negatively constrained by the quality and coverage of the training dataset. In contrast RL-trained G&CNETs, beyond demonstrating a superior adaptability to stochastic conditions, can also discover solutions that improve upon suboptimal expert demonstrations, sometimes revealing globally optimal strategies that eluded the generation of training samples.