LMFPPO-UBP: Local Mean Field Proximal Policy Optimization with Unbalanced Punishment for Spatial Public Goods Games
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
Spatial public goods games are characterized by high-dimensional state spaces and localized externalities, which pose significant challenges for achieving stable and widespread cooperation. Traditional approaches often struggle to effectively capture neighborhood-level strategic interactions and dynamically align individual incentives with collective welfare. To resolve this issue, this paper introduces a novel intelligent decision-making framework called Local Mean-Field Proximal Policy Optimization with Unbalanced Punishment (LMFPPO-UBP). The conventional mean field concept is reformulated as a socio-statistical sensor embedded directly into the policy gradient space of deep reinforcement learning, allowing agents to adapt their strategies based on mesoscale neighborhood dynamics. Additionally, an unbalanced punishment mechanism is integrated to penalize defectors proportionally to the local density of cooperators, thereby reshaping the payoff structures without imposing direct costs on cooperative agents. Experimental results demonstrate that the LMFPPO-UBP promotes rapid and stable global cooperation even under low enhancement factors, consistently outperforming baseline methods such as Q-learning and Fermi update rules. Statistical analyses further validate the framework's effectiveness in lowering the cooperation threshold and achieving better coordinated outcomes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 38 references that are in the map. First posted on arXiv 21 Feb 2026.
Builds on
7 of 38 references.
References · 7
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
- 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
- 2025Bottom-Up Reputation Promotes Cooperation with Multi-Agent Reinforcement Learningcited by 5
- 2020Blocking defector invasion by focusing on the most successful partnercited by 2
- 2024Learning Nudges for Conditional Cooperation: A Multi-Agent Reinforcement Learning Modelcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025TUC-PPO: Team Utility-Constrained Proximal Policy Optimization for Spatial Public Goods Games
- 2025GRPO-GCC: Enhancing Cooperation in Spatial Public Goods Games via Group Relative Policy Optimization with Global Cooperation Constraint ↗
- 2025MAPPO-LCR: Multi-Agent Proximal Policy Optimization with Local Cooperation Reward in Spatial Public Goods Games ↗
- 2025PPO-ACT: Proximal Policy Optimization with Adversarial Curriculum Transfer for Spatial Public Goods Games
- 2024Reinforcement Learning for Finite Space Mean-Field Type Games
- 2025A Game-Theoretic Spatio-Temporal Reinforcement Learning Framework for Collaborative Public Resource Allocation
- 2024Efficient and Scalable Deep Reinforcement Learning for Mean Field Control Games
- 2025Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games