# Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing

arXiv: https://arxiv.org/abs/2309.08477 · HTML: https://mlsanity.com/paper/2309.08477/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [RLlib: Abstractions for Distributed Reinforcement Learning](https://mlsanity.com/paper/1712.09381/) (2017)
- [Federated Learning: A Signal Processing Perspective](https://mlsanity.com/paper/2103.17150/) (2021)
- [PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning](https://mlsanity.com/paper/2001.05012/) (2020)
- [Policy Design for Active Sequential Hypothesis Testing using Deep Learning](https://mlsanity.com/paper/1810.04859/) (2018)
- [Deep Actor-Critic Reinforcement Learning for Anomaly Detection](https://mlsanity.com/paper/1908.10755/) (2019)
- [Distributed Learning Algorithms for Spectrum Sharing in Spatial Random Access Wireless Networks](https://mlsanity.com/paper/1507.05664/) (2015)
- [Anomaly Detection Under Controlled Sensing Using Actor-Critic Reinforcement Learning](https://mlsanity.com/paper/2006.01044/) (2020)
- [Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approach](https://mlsanity.com/paper/2303.15544/) (2023)
- [Controlled Sensing for Sequential Multihypothesis Testing with Controlled Markovian Observations and Non-Uniform Control Cost](https://mlsanity.com/paper/1310.1844/) (2013)
- [Optimal Index Policies for Anomaly Localization in Resource-Constrained Cyber Systems](https://mlsanity.com/paper/1405.4098/) (2014)
- [Anomaly Search with Multiple Plays under Delay and Switching Costs](https://mlsanity.com/paper/2108.03082/) (2021)
- [Signaling in sensor networks for sequential detection](https://mlsanity.com/paper/1403.3126/) (2014)
- [Joint Sequential Detection and Isolation for Dependent Data Streams](https://mlsanity.com/paper/2207.00120/) (2022)

## Built on by (most cited first, 4 of 4)

- [Searching for a Hidden Markov Anomaly over Multiple Processes](https://mlsanity.com/paper/2506.17108/) (2025)
- [Control-Optimized Deep Reinforcement Learning for Artificially Intelligent Autonomous Systems](https://mlsanity.com/paper/2507.00268/) (2025)
- [On Cost-Aware Designs for Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.19067/) (2025)
- [ICMAPE: In-Context Multiagent Pure Exploration](https://mlsanity.com/paper/2609.33986/) (2026)

## Abstract (arXiv)

We consider a decentralized formulation of the active hypothesis testing (AHT) problem, where multiple agents gather noisy observations from the environment with the purpose of identifying the correct hypothesis. At each time step, agents have the option to select a sampling action. These different actions result in observations drawn from various distributions, each associated with a specific hypothesis. The agents collaborate to accomplish the task, where message exchanges between agents are allowed over a rate-limited communications channel. The objective is to devise a multi-agent policy that minimizes the Bayes risk. This risk comprises both the cost of sampling and the joint terminal cost incurred by the agents upon making a hypothesis declaration. Deriving optimal structured policies for AHT problems is generally mathematically intractable, even in the context of a single agent. As a result, recent efforts have turned to deep learning methodologies to address these problems, which have exhibited significant success in single-agent learning scenarios. In this paper, we tackle the multi-agent AHT formulation by introducing a novel algorithm rooted in the framework of deep multi-agent reinforcement learning. This algorithm, named Multi-Agent Reinforcement Learning for AHT (MARLA), operates at each time step by having each agent map its state to an action (sampling rule or stopping rule) using a trained deep neural network with the goal of minimizing the Bayes risk. We present a comprehensive set of experimental results that effectively showcase the agents' ability to learn collaborative strategies and enhance performance using MARLA. Furthermore, we demonstrate the superiority of MARLA over single-agent learning approaches. Finally, we provide an open-source implementation of the MARLA framework, for the benefit of researchers and developers in related domains.