# Q-Learning-Driven Adaptive Rewiring for Cooperative Control in Heterogeneous Networks

arXiv: https://arxiv.org/abs/2509.01057 · HTML: https://mlsanity.com/paper/2509.01057/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 76 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 8 of 8)

- [Statistical physics of human cooperation](https://mlsanity.com/paper/1705.07161/) (2017)
- [Social Clustering in Epidemic Spread on Coevolving Networks](https://mlsanity.com/paper/1707.04933/) (2017)
- [Evolutionary Multi-agent Reinforcement Learning in Group Social Dilemmas](https://mlsanity.com/paper/2411.10459/) (2024)
- [SDN-based Resource Allocation in Edge and Cloud Computing Systems: An Evolutionary Stackelberg Differential Game Approach](https://mlsanity.com/paper/2109.12543/) (2021)
- [Evolutionary prisoner's dilemma games coevolving on adaptive networks](https://mlsanity.com/paper/1702.05119/) (2017)
- [Evolutionary dynamics of cooperation in neutral populations](https://mlsanity.com/paper/1712.02321/) (2017)
- [Mercenary punishment in structured populations](https://mlsanity.com/paper/2111.04480/) (2021)
- [Strategy evolution on dynamic networks](https://mlsanity.com/paper/2301.11982/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Collective Variational Principle Unifying Bayesian Inference, Game Theory, and Thermodynamics](https://mlsanity.com/paper/2604.27942/) (2026)

## Abstract (arXiv)

Cooperation emergence in multi-agent systems represents a fundamental statistical physics problem where microscopic learning rules drive macroscopic collective behavior transitions. We propose a Q-learning-based variant of adaptive rewiring that builds on mechanisms studied in the literature. This method combines temporal difference learning with network restructuring so that agents can optimize strategies and social connections based on interaction histories. Through neighbor-specific Q-learning, agents develop sophisticated partnership management strategies that enable cooperator cluster formation, creating spatial separation between cooperative and defective regions. Using power-law networks that reflect real-world heterogeneous connectivity patterns, we evaluate emergent behaviors under varying rewiring constraint levels, revealing distinct cooperation patterns across parameter space rather than sharp thermodynamic transitions. Our systematic analysis identifies three behavioral regimes: a permissive regime (low constraints) enabling rapid cooperative cluster formation, an intermediate regime with sensitive dependence on dilemma strength, and a patient regime (high constraints) where strategic accumulation gradually optimizes network structure. Simulation results show that while moderate constraints create transition-like zones that suppress cooperation, fully adaptive rewiring enhances cooperation levels through systematic exploration of favorable network configurations. Quantitative analysis reveals that increased rewiring frequency drives large-scale cluster formation with power-law size distributions. Our results establish a new paradigm for understanding intelligence-driven cooperation pattern formation in complex adaptive systems, revealing how machine learning serves as an alternative driving force for spontaneous organization in multi-agent networks.