mlsanityData to 14 Oct 2026

Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks

arXiv 2408.10556 ↗First posted 20 Aug 2024cs.AI cs.LGcited by 11

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

The advancement of Offline Reinforcement Learning (RL) and Offline Multi-Agent Reinforcement Learning (MARL) critically depends on the availability of high-quality, pre-collected offline datasets that represent real-world complexities and practical applications. However, existing datasets often fall short in their simplicity and lack of realism. To address this gap, we propose Hokoff, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research. This data is derived from Honor of Kings, a recognized Multiplayer Online Battle Arena (MOBA) game known for its intricate nature, closely resembling real-life situations. Utilizing this framework, we benchmark a variety of offline RL and offline MARL algorithms. We also introduce a novel baseline algorithm tailored for the inherent hierarchical action space of the game. We reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 50 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

36 of 50 references.

Closest ancestors · 19

Same direction, newest first

  1. 2023Supported Trust Region Optimization for Offline Reinforcement Learning15 Nov
  2. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning22 Sep
  3. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learning5 Feb
  4. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  5. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  6. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learning16 Sep
  7. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning19 Aug
  8. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble1 Jul
  9. 2021Boosting Offline Reinforcement Learning with Residual Generative Modeling19 Jun
  10. 2021Bellman-consistent Pessimism for Offline Reinforcement Learning13 Jun
  11. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  12. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning17 May
7 more
  1. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism22 Mar
  2. 2020Is Pessimism Provably Efficient for Offline RL?30 Dec
  3. 2020Provably Good Batch Reinforcement Learning Without Great Exploration16 Jul
  4. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  7. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 17

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  6. 2019The StarCraft Multi-Agent Challengecited by 597
  7. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  8. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  9. 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
  10. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  11. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  12. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
5 more
  1. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 61
  2. 2021Offline Decentralized Multi-Agent Reinforcement Learningcited by 33
  3. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 28
  4. 2018Multi-vehicle Flocking Control with Deep Deterministic Policy Gradient Methodcited by 7
  5. 2021Learning Diverse Policies in MOBA Games via Macro-Goalscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 3.

11 papers.

  1. 2025FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Expertscited by 35
  2. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
  3. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
  4. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 12
  5. 2025Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learningcited by 11
  6. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 4
  7. 2025What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinkingcited by 3
  8. 2025Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learningcited by 2
  9. 2026Mastering PokeGym: Graph-Guided Multimodal Evolution at Test Timecited by 1
  10. 2026Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RLcited by 0
  11. 2026Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  2. 2023AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
  3. 2022Offline Equilibrium Finding
  4. 2025A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
  5. 2024Offline Fictitious Self-Play for Competitive Games
  6. 2024Mini Honor of Kings: A Lightweight Environment for Multi-Agent Reinforcement Learning
  7. 2024A Benchmark Environment for Offline Reinforcement Learning in Racing Games
  8. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining