Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
In the realm of competitive gaming, 3D first-person shooter (FPS) games have gained immense popularity, prompting the development of game AI systems to enhance gameplay. However, deploying game AI in practical scenarios still poses challenges, particularly in large-scale and complex FPS games. In this paper, we focus on the practical deployment of game AI in the online multiplayer competitive 3D FPS game called Arena Breakout, developed by Tencent Games. We propose a novel gaming AI system named Private Military Company Agent (PMCA), which is interactable within a large game map and engages in combat with players while utilizing tactical advantages provided by the surrounding terrain. To address the challenges of navigation and combat in modern 3D FPS games, we introduce a method that combines navigation mesh (Navmesh) and shooting-rule with deep reinforcement learning (NSRL). The integration of Navmesh enhances the agent's global navigation capabilities while shooting behavior is controlled using rule-based methods to ensure controllability. NSRL employs a DRL model to predict when to enable the navigation mesh, resulting in a diverse range of behaviors for the game AI. Customized rewards for human-like behaviors are also employed to align PMCA's behavior with that of human players.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 15 references that are in the map. First posted on arXiv 7 Oct 2024.
Builds on
10 of 15 references.
References · 10
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
- 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
- 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
- 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
- 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 50
- 2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22
- 2022WILD-SCAV: Benchmarking FPS Gaming AI on Unity3D-based Environmentscited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 4.
5 papers.
- 2025Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Modelscited by 7
- 2025Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approachcited by 2
- 2024Human-like Bots for Tactical Shooters Using Compute-Efficient Sensorscited by 1
- 2025Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Explorationcited by 1
- 2025Combining Reinforcement Learning and Behavior Trees for NPCs in Video Games with AMD Scholacited by 0
- 2025Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach27 Oct 2025
- 2025Combining Reinforcement Learning and Behavior Trees for NPCs in Video Games with AMD Schola15 Oct 2025
- 2025Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models29 Aug 2025
- 2025Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Exploration17 Mar 2025
- 2024Human-like Bots for Tactical Shooters Using Compute-Efficient Sensors30 Dec 2024
Similar, not cited
Close in wording, no citation either way
- 2025Agents Play Thousands of 3D Video Games
- 2025A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
- 2025Reinforcement Learning Agent for a 2D Shooter Game
- 2026Player-1: Learning to Play from Synthetic Experience
- 2025A Hierarchical Hybrid AI Approach: Integrating Deep Reinforcement Learning and Scripted Agents in Combat Simulations
- 2025Cultivating Game Sense for Yourself: Making VLMs Gaming Experts
- 2026NitroGen: An Open Foundation Model for Generalist Gaming Agents
- 2025Enhancing Player Enjoyment with a Two-Tier DRL and LLM-Based Agent System for Fighting Games