# Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards

arXiv: https://arxiv.org/abs/2302.05838 · HTML: https://mlsanity.com/paper/2302.05838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Reverse Curriculum Generation for Reinforcement Learning](https://mlsanity.com/paper/1707.05300/) (2017)
- [Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO](https://mlsanity.com/paper/2005.12729/) (2020)
- [Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks](https://mlsanity.com/paper/1802.03796/) (2018)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)
- [Curriculum Learning of Multiple Tasks](https://mlsanity.com/paper/1412.1353/) (2014)
- [A Competence-aware Curriculum for Visual Concepts Learning via Question Answering](https://mlsanity.com/paper/2007.01499/) (2020)
- [Accuracy-based Curriculum Learning in Deep Reinforcement Learning](https://mlsanity.com/paper/1806.09614/) (2018)
- [Coarse-to-Fine Curriculum Learning](https://mlsanity.com/paper/2106.04072/) (2021)
- [Efficient Multitask Feature and Relationship Learning](https://mlsanity.com/paper/1702.04423/) (2017)
- [Multi Pseudo Q-learning Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles](https://mlsanity.com/paper/1909.03204/) (2019)
- [Supervised Machine Learning for Effective Missile Launch Based on Beyond Visual Range Air Combat Simulations](https://mlsanity.com/paper/2207.04188/) (2022)
- [Curriculum By Smoothing](https://mlsanity.com/paper/2003.01367/) (2020)
- [Reinforcement Learning based Air Combat Maneuver Generation](https://mlsanity.com/paper/2201.05528/) (2022)
- [Homotopy Based Reinforcement Learning with Maximum Entropy for Autonomous Air Combat](https://mlsanity.com/paper/2112.01328/) (2021)

## Abstract (arXiv)

Reinforcement learning is an effective way to solve the decision-making problems. It is a meaningful and valuable direction to investigate autonomous air combat maneuver decision-making method based on reinforcement learning. However, when using reinforcement learning to solve the decision-making problems with sparse rewards, such as air combat maneuver decision-making, it costs too much time for training and the performance of the trained agent may not be satisfactory. In order to solve these problems, the method based on curriculum learning is proposed. First, three curricula of air combat maneuver decision-making are designed: angle curriculum, distance curriculum and hybrid curriculum. These courses are used to train air combat agents respectively, and compared with the original method without any curriculum. The training results show that angle curriculum can increase the speed and stability of training, and improve the performance of the agent; distance curriculum can increase the speed and stability of agent training; hybrid curriculum has a negative impact on training, because it makes the agent get stuck at local optimum. The simulation results show that after training, the agent can handle the situations where targets come from different directions, and the maneuver decision results are consistent with the characteristics of missile.