# BVR Gym: A Reinforcement Learning Environment for Beyond-Visual-Range Air Combat

arXiv: https://arxiv.org/abs/2403.17533 · HTML: https://mlsanity.com/paper/2403.17533/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising](https://mlsanity.com/paper/1808.00720/) (2018)
- [A Survey of Behavior Trees in Robotics and AI](https://mlsanity.com/paper/2005.05842/) (2020)
- [Reinforcement Learning for UAV Attitude Control](https://mlsanity.com/paper/1804.04154/) (2018)
- [CropGym: a Reinforcement Learning Environment for Crop Management](https://mlsanity.com/paper/2104.04326/) (2021)
- [Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization](https://mlsanity.com/paper/1911.05478/) (2019)
- [Behavior Trees in Robot Control Systems](https://mlsanity.com/paper/2203.13083/) (2022)
- [On modularity in reactive control architectures, with an application to formal verification](https://mlsanity.com/paper/2008.12515/) (2020)
- [Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trials](https://mlsanity.com/paper/2105.00990/) (2021)
- [Engagement Decision Support for Beyond Visual Range Air Combat](https://mlsanity.com/paper/2111.03059/) (2021)

## Built on by (most cited first, 3 of 3)

- [A survey of air combat behavior modeling using machine learning](https://mlsanity.com/paper/2404.13954/) (2024)
- [Towards Human Engagement with Realistic AI Combat Pilots](https://mlsanity.com/paper/2509.26002/) (2025)
- [BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning](https://mlsanity.com/paper/2608.25419/) (2026)

## Abstract (arXiv)

Creating new air combat tactics and discovering novel maneuvers can require numerous hours of expert pilots' time. Additionally, for each different combat scenario, the same strategies may not work since small changes in equipment performance may drastically change the air combat outcome. For this reason, we created a reinforcement learning environment to help investigate potential air combat tactics in the field of beyond-visual-range (BVR) air combat: the BVR Gym. This type of air combat is important since long-range missiles are often the first weapon to be used in aerial combat. Some existing environments provide high-fidelity simulations but are either not open source or are not adapted to the BVR air combat domain. Other environments are open source but use less accurate simulation models. Our work provides a high-fidelity environment based on the open-source flight dynamics simulator JSBSim and is adapted to the BVR air combat domain. This article describes the building blocks of the environment and some use cases.