# Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1710.10363 · HTML: https://mlsanity.com/paper/1710.10363/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2017.

## Built on by (most cited first, 5 of 5)

- [Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functions](https://mlsanity.com/paper/2202.13046/) (2022)
- [Fully Distributed Actor-Critic Architecture for Multitask Deep Reinforcement Learning](https://mlsanity.com/paper/2110.12306/) (2021)
- [Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR](https://mlsanity.com/paper/2504.20927/) (2025)
- [Domain-driven Metrics for Reinforcement Learning: A Case Study on Epidemic Control using Agent-based Simulation](https://mlsanity.com/paper/2508.05154/) (2025)
- [Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspective](https://mlsanity.com/paper/2510.09937/) (2025)

## Abstract (arXiv)

We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named textitDiff-DAC, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average for the whole set of tasks. During the learning process, agents communicate their value-policy parameters to their neighbors, diffusing the information across the network, so that they converge to a common policy, with no need for a central node. The method is scalable, since the computational and communication costs per agent grow with its number of neighbors. We derive Diff-DAC's from duality theory and provide novel insights into the standard actor-critic framework, showing that it is actually an instance of the dual ascent method that approximates the solution of a linear program. Experiments suggest that Diff-DAC can outperform the single previous distributed MRL approach (i.e., Dist-MTLPS) and even the centralized architecture.