# Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents

arXiv: https://arxiv.org/abs/1912.00498 · HTML: https://mlsanity.com/paper/1912.00498/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2019.

## Built on by (most cited first, 20 of 20)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [A Survey on Agent Workflow -- Status and Future](https://mlsanity.com/paper/2508.01186/) (2025)
- [Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees](https://mlsanity.com/paper/2110.05597/) (2021)
- [MARL with General Utilities via Decentralized Shadow Reward Actor-Critic](https://mlsanity.com/paper/2106.00543/) (2021)
- [Kolmogorov-Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Bandits](https://mlsanity.com/paper/2105.14586/) (2021)
- [Towards Learning Generalizable Driving Policies from Restricted Latent Representations](https://mlsanity.com/paper/2111.03688/) (2021)
- [Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2108.04087/) (2021)
- [Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learning](https://mlsanity.com/paper/2501.04879/) (2025)
- [Matrix Low-Rank Approximation For Policy Gradient Methods](https://mlsanity.com/paper/2405.17626/) (2024)
- [FlowRL: Flow-Augmented Few-Shot Reinforcement Learning for Semi-Structured Sensor Data](https://mlsanity.com/paper/2409.14178/) (2024)
- [Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market](https://mlsanity.com/paper/2411.16519/) (2024)
- [Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programming](https://mlsanity.com/paper/2110.12929/) (2021)
- [Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2111.12961/) (2021)
- [Feature-Aware Task-to-Core Allocation in Embedded Multi-core Platforms via Statistical Learning](https://mlsanity.com/paper/2502.15716/) (2025)
- [Cooperative Actor-Critic via TD Error Aggregation](https://mlsanity.com/paper/2207.12533/) (2022)
- [Continuous-Time Distributed Dynamic Programming for Networked Multi-Agent Markov Decision Processes](https://mlsanity.com/paper/2307.16706/) (2023)
- [Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions](https://mlsanity.com/paper/2601.00321/) (2026)
- [ZeroDVFS: Zero-Shot LLM-Guided Core and Frequency Allocation for Embedded Platforms](https://mlsanity.com/paper/2601.08166/) (2026)

## Abstract (arXiv)

This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate. We provide an overview of this emerging field, with an emphasis on the decentralized setting under different coordination protocols. We highlight the evolution of reinforcement learning algorithms from single-agent to multi-agent systems, from a distributed optimization perspective, and conclude with future directions and challenges, in the hope to catalyze the growing synergy among distributed optimization, signal processing, and reinforcement learning communities.