Guidance Design for Escape Flight Vehicle Using Evolution Strategy Enhanced Deep Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
Guidance commands of flight vehicles are a series of data sets with fixed time intervals, thus guidance design constitutes a sequential decision problem and satisfies the basic conditions for using deep reinforcement learning (DRL). In this paper, we consider the scenario where the escape flight vehicle (EFV) generates guidance commands based on DRL and the pursuit flight vehicle (PFV) generates guidance commands based on the proportional navigation method. For the EFV, the objective of the guidance design entails progressively maximizing the residual velocity, subject to the constraint imposed by the given evasion distance. Thus an irregular dynamic max-min problem of extremely large-scale is formulated, where the time instant when the optimal solution can be attained is uncertain and the optimum solution depends on all the intermediate guidance commands generated before. For solving this problem, a two-step strategy is conceived. In the first step, we use the proximal policy optimization (PPO) algorithm to generate the guidance commands of the EFV. The results obtained by PPO in the global search space are coarse, despite the fact that the reward function, the neural network parameters and the learning rate are designed elaborately. Therefore, in the second step, we propose to invoke the evolution strategy (ES) based algorithm, which uses the result of PPO as the initial value, to further improve the quality of the solution by searching in the local space. Simulation results demonstrate that the proposed guidance design method based on the PPO algorithm is capable of achieving a residual velocity of 67.24 m/s, higher than the residual velocities achieved by the benchmark soft actor-critic and deep deterministic policy gradient algorithms. Furthermore, the proposed ES-enhanced PPO algorithm outperforms the PPO algorithm by 2.7%, achieving a residual velocity of 69.04 m/s.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 44 references that are in the map. First posted on arXiv 4 May 2024.
Builds on
10 of 44 references.
References · 10
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2015Trust Region Policy Optimizationcited by 2,580
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2019Truly Proximal Policy Optimizationcited by 69
- 2020Supervised Learning Achieves Human-Level Performance in MOBA Games: A Case Study of Honor of Kingscited by 18
- 2020COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learningcited by 17
- 2019Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Controlcited by 2
- 2019Reinforcement Learning for Angle-Only Intercept Guidance of Maneuvering Targetscited by 2
- 2021Collision Avoidance for Unmanned Aerial Vehicles in the Presence of Static and Moving Obstaclescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors
- 2024Deep Reinforcement Learning with Enhanced PPO for Safe Mobile Robot Navigation
- 2025Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach
- 2024Online Planning for Multi-UAV Pursuit-Evasion in Unknown Environments Using Deep Reinforcement Learning
- 2020Exploration-efficient Deep Reinforcement Learning with Demonstration Guidance for Robot Control
- 2026Adversarial Reinforcement Learning for Robust Planetary Landing
- 2025The Emergence of Deep Reinforcement Learning for Path Planning
- 2024NavRL: Learning Safe Flight in Dynamic Environments