# The Confluence of Networks, Games and Learning

arXiv: https://arxiv.org/abs/2105.08158 · HTML: https://mlsanity.com/paper/2105.08158/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 131 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 37)

- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [A unified view of entropy-regularized Markov decision processes](https://mlsanity.com/paper/1705.07798/) (2017)
- [Cycles in adversarial regularized learning](https://mlsanity.com/paper/1709.02738/) (2017)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization](https://mlsanity.com/paper/2011.00364/) (2020)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity](https://mlsanity.com/paper/2007.07461/) (2020)
- [Learning in games with continuous action sets and unknown payoff functions](https://mlsanity.com/paper/1608.07310/) (2016)
- [On Gradient-Based Learning in Continuous Games](https://mlsanity.com/paper/1804.05464/) (2018)
- [Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization](https://mlsanity.com/paper/1806.00877/) (2018)
- [Bandit learning in concave N-person games](https://mlsanity.com/paper/1810.01925/) (2018)
- [Fictitious play in zero-sum stochastic games](https://mlsanity.com/paper/2010.04223/) (2020)
- [Riemannian game dynamics](https://mlsanity.com/paper/1603.09173/) (2016)
- [A Dynamic Games Approach to Proactive Defense Strategies against Advanced Persistent Threats in Cyber-Physical Systems](https://mlsanity.com/paper/1906.09687/) (2019)
- [Global Convergence of Policy Gradient for Sequential Zero-Sum Linear Quadratic Dynamic Games](https://mlsanity.com/paper/1911.04672/) (2019)
- [Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity](https://mlsanity.com/paper/2101.01041/) (2021)
- [On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning](https://mlsanity.com/paper/1908.08578/) (2019)
- [Blackwell Online Learning for Markov Decision Processes](https://mlsanity.com/paper/2012.14043/) (2020)
- [On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games](https://mlsanity.com/paper/1808.04464/) (2018)
- [A Passivity-Based Approach to Nash Equilibrium Seeking over Networks](https://mlsanity.com/paper/1705.02424/) (2017)
- [Combining No-regret and Q-learning](https://mlsanity.com/paper/1910.03094/) (2019)
- [Stationary Markov Perfect Equilibria in Discounted Stochastic Games](https://mlsanity.com/paper/1311.1562/) (2013)
- [Learning Stationary Correlated Equilibria in Constrained General-Sum Stochastic Games](https://mlsanity.com/paper/1402.1027/) (2014)
- [Regular Potential Games](https://mlsanity.com/paper/1707.06466/) (2017)
- [Modeling, Analysis, and Mitigation of Dynamic Botnet Formation in Wireless IoT Networks](https://mlsanity.com/paper/1901.00741/) (2018)
- [A Differential Game Approach to Decentralized Virus-Resistant Weight Adaptation Policy over Complex Networks](https://mlsanity.com/paper/1905.02237/) (2019)

## Built on by (most cited first, 26 of 26)

- [Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applications](https://mlsanity.com/paper/2206.07922/) (2022)
- [Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Games](https://mlsanity.com/paper/2402.18781/) (2024)
- [Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation](https://mlsanity.com/paper/2210.03209/) (2022)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement Learning](https://mlsanity.com/paper/2306.06766/) (2023)
- [Game Theory Meets LLM and Agentic AI: Reimagining Cybersecurity for the Age of Intelligent Threats](https://mlsanity.com/paper/2507.10621/) (2025)
- [Meta Stackelberg Game: Robust Federated Learning against Adaptive and Mixed Poisoning Attacks](https://mlsanity.com/paper/2410.17431/) (2024)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Symbiotic Game and Foundation Models for Cyber Deception Operations in Strategic Cyber Warfare](https://mlsanity.com/paper/2403.10570/) (2024)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [A First Order Meta Stackelberg Method for Robust Federated Learning](https://mlsanity.com/paper/2306.13800/) (2023)
- [Multi-level Traffic-Responsive Tilt Camera Surveillance through Predictive Correlated Online Learning](https://mlsanity.com/paper/2408.02208/) (2024)
- [Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis](https://mlsanity.com/paper/2208.00081/) (2022)
- [In-Context Reinforcement Learning via Communicative World Models](https://mlsanity.com/paper/2508.06659/) (2025)
- [Is Stochastic Mirror Descent Vulnerable to Adversarial Delay Attacks? A Traffic Assignment Resilience Study](https://mlsanity.com/paper/2304.01161/) (2023)
- [Self-Confirming Transformer for Belief-Conditioned Adaptation in Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.04579/) (2023)
- [From Texts to Shields: Convergence of Large Language Models and Cybersecurity](https://mlsanity.com/paper/2505.00841/) (2025)
- [Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation](https://mlsanity.com/paper/2506.22365/) (2025)
- [Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations](https://mlsanity.com/paper/2512.22883/) (2025)
- [Learning from Response not Preference: A Stackelberg Approach for LLM Detoxification using Non-parallel Data](https://mlsanity.com/paper/2410.20298/) (2024)
- [Digital Twin-based Driver Risk-Aware Intelligent Mobility Analytics for Urban Transportation Management](https://mlsanity.com/paper/2407.15025/) (2024)
- [Commitment with Signaling under Double-sided Information Asymmetry](https://mlsanity.com/paper/2212.11446/) (2022)
- [Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environments](https://mlsanity.com/paper/2309.02328/) (2023)
- [Distributed Online Convex Optimization with Nonseparable Costs and Constraints](https://mlsanity.com/paper/2602.10452/) (2026)
- [Agentic Incident Response through Digital Twin-Enhanced Multiscale Planning](https://mlsanity.com/paper/2608.02422/) (2026)
- [AdaFair-MARL: Enforcing Adaptive Fairness Constraints in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2511.14135/) (2025)

## Abstract (arXiv)

Recent years have witnessed significant advances in technologies and services in modern network applications, including smart grid management, wireless communication, cybersecurity as well as multi-agent autonomous systems. Considering the heterogeneous nature of networked entities, emerging network applications call for game-theoretic models and learning-based approaches in order to create distributed network intelligence that responds to uncertainties and disruptions in a dynamic or an adversarial environment. This paper articulates the confluence of networks, games and learning, which establishes a theoretical underpinning for understanding multi-agent decision-making over networks. We provide an selective overview of game-theoretic learning algorithms within the framework of stochastic approximation theory, and associated applications in some representative contexts of modern network systems, such as the next generation wireless communication networks, the smart grid and distributed machine learning. In addition to existing research works on game-theoretic learning over networks, we highlight several new angles and research endeavors on learning in games that are related to recent developments in artificial intelligence. Some of the new angles extrapolate from our own research interests. The overall objective of the paper is to provide the reader a clear picture of the strengths and challenges of adopting game-theoretic learning methods within the context of network systems, and further to identify fruitful future research directions on both theoretical and applied studies.