mlsanityData to 14 Oct 2026

A High-Throughput Compute-Efficient POMDP Hide-And-Seek-Engine (HASE) for Multi-Agent Operations

arXiv 2604.27162 ↗First posted 29 Apr 2026cs.MA cs.LG cs.PFcited by 0

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) algorithms exhibit high sample complexity, particularly when applied to Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). As a response, projects such as SampleFactory, EnvPool, Brax, and IsaacLab migrate parallel execution of classic environments such as MuJoCo and Atari into C++ thread pools or the GPU to decrease the computational cost of environment steps. We are interested in optimizing the decision-level of human-AI joint operations, so we introduce a compute-efficient Dec-POMDP engine natively architected in C++ called Hide-And-Seek-Engine. By employing Data-Oriented Design (DOD) principles, explicit 64-byte cache-line alignment to remove false sharing, and a zero-copy PyTorch memory bridge using pinned memory and Direct Memory Access (DMA), our engine sustains throughput of up to 33,000,000 steps per second (SPS) in a single-agent, 1024-environment, decentralized observations on an AMD Ryzen 9950X (16 cores). Ten agents reduces FPS to 7M SPS with generating random actions contributing 1/3rd the total runtime for reference. The engine achieves a throughput increase of approximately 3,500× over the baseline single threaded vectorized NumPy implementation and successfully trains cooperative multi-agent policies via PPO, DQN, and SAC in minutes, validating both its performance and generality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

24 of 38 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  10. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  11. 2018Distributed Prioritized Experience Replaycited by 269
  12. 2018Unity: A General Platform for Intelligent Agentscited by 236
12 more
  1. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  2. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  3. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  5. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
  6. 2024Simplifying Deep Temporal Difference Learningcited by 78
  7. 2025Robust Autonomy Emerges from Self-Playcited by 67
  8. 2020Sample-Efficient Reinforcement Learning of Undercomplete POMDPscited by 66
  9. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 62
  10. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 24
  11. 2020Bandwidth-Aware Page Placement in NUMAcited by 1
  12. 2025Impact of Data-Oriented and Object-Oriented Design on Performance and Cache Utilization with Artificial Intelligence Algorithms in Multi-Threaded CPUscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling
  2. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
  3. 2026EvolveGen: Algorithmic Level Hardware Model Checking Benchmark Generation through Reinforcement Learning
  4. 2026Probing Dec-POMDP Reasoning in Cooperative MARL
  5. 2026TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
  6. 2026PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
  7. 2026CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning
  8. 2026StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning