mlsanityData to 14 Oct 2026

A survey of air combat behavior modeling using machine learning

arXiv 2404.13954 ↗First posted 22 Apr 2024cs.LG cs.AI cs.MAcited by 3

Not on the map. Why?

Abstract (arXiv)

With the recent advances in machine learning, creating agents that behave realistically in simulated air combat has become a growing field of interest. This survey explores the application of machine learning techniques for modeling air combat behavior, motivated by the potential to enhance simulation-based pilot training. Current simulated entities tend to lack realistic behavior, and traditional behavior modeling is labor-intensive and prone to loss of essential domain knowledge between development steps. Advancements in reinforcement learning and imitation learning algorithms have demonstrated that agents may learn complex behavior from data, which could be faster and more scalable than manual methods. Yet, making adaptive agents capable of performing tactical maneuvers and operating weapons and sensors still poses a significant challenge. The survey examines applications, behavior model types, prevalent machine learning methods, and the technical and human challenges in developing adaptive and realistically behaving agents. Another challenge is the transfer of agents from learning environments to military simulation systems and the consequent demand for standardization. Four primary recommendations are presented regarding increased emphasis on beyond-visual-range scenarios, multi-agent machine learning and cooperation, utilization of hierarchical behavior models, and initiatives for standardization and research collaboration. These recommendations aim to address current issues and guide the development of more comprehensive, adaptable, and realistic machine learning-based behavior models for air combat applications.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 28 of its 154 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

28 of 154 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2015Trust Region Policy Optimizationcited by 2,580
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2016Generative Adversarial Imitation Learningcited by 1,597
  9. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  10. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  11. 2015Prioritized Experience Replaycited by 1,099
  12. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
16 more
  1. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  2. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  3. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  4. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 242
  5. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  6. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
  7. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  8. 2020A Survey of Behavior Trees in Robotics and AIcited by 55
  9. 2019Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Surveycited by 47
  10. 2022A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combatcited by 10
  11. 2021Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trialscited by 8
  12. 2021Engagement Decision Support for Beyond Visual Range Air Combatcited by 5
  13. 2023Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuveringcited by 5
  14. 2019Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trailscited by 4
  15. 2024BVR Gym: A Reinforcement Learning Environment for Beyond-Visual-Range Air Combatcited by 3
  16. 2023Learning Environment for the Air Domain (LEAD)cited by 1

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Training Environment for High Performance Reinforcement Learningcited by 0
  2. 2025Enhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learningcited by 0
  3. 2025Explaining Strategic Decisions in Multi-Agent Reinforcement Learning for Aerial Combat Tacticscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning
  2. 2025Coordinated Strategies in Realistic Air Combat by Hierarchical Multi-Agent Reinforcement Learning
  3. 2024Scaling Intelligent Agents in Combat Simulations for Wargaming
  4. 2025A Hierarchical Hybrid AI Approach: Integrating Deep Reinforcement Learning and Scripted Agents in Combat Simulations
  5. 2025Towards Human Engagement with Realistic AI Combat Pilots
  6. 2026BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning
  7. 2021Adaptive Synthetic Characters for Military Training
  8. 2025Light Aircraft Game : Basic Implementation and training results analysis