# Coordinated Strategies in Realistic Air Combat by Hierarchical Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2510.11474 · HTML: https://mlsanity.com/paper/2510.11474/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Simple Policy Optimization](https://mlsanity.com/paper/2401.16025/) (2024)
- [LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios](https://mlsanity.com/paper/2310.08348/) (2023)
- [When Maximum Entropy Misleads Policy Optimization](https://mlsanity.com/paper/2506.05615/) (2025)
- [Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering](https://mlsanity.com/paper/2309.11247/) (2023)
- [A Hierarchical Reinforcement Learning Framework for Multi-UAV Combat Using Leader-Follower Strategy](https://mlsanity.com/paper/2501.13132/) (2025)

## Built on by (most cited first, 1 of 1)

- [Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.09212/) (2026)

## Abstract (arXiv)

Achieving mission objectives in a realistic simulation of aerial combat is highly challenging due to imperfect situational awareness and nonlinear flight dynamics. In this work, we introduce a novel 3D multi-agent air combat environment and a Hierarchical Multi-Agent Reinforcement Learning framework to tackle these challenges. Our approach combines heterogeneous agent dynamics, curriculum learning, league-play, and a newly adapted training algorithm. To this end, the decision-making process is organized into two abstraction levels: low-level policies learn precise control maneuvers, while high-level policies issue tactical commands based on mission objectives. Empirical results show that our hierarchical approach improves both learning efficiency and combat performance in complex dogfight scenarios.