# Inverse Reinforcement Learning in Swarm Systems

arXiv: https://arxiv.org/abs/1602.05450 · HTML: https://mlsanity.com/paper/1602.05450/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2016.

## Built on by (most cited first, 18 of 18)

- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [Multi-Agent Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1907.13220/) (2019)
- [Towards Distraction-Robust Active Visual Tracking](https://mlsanity.com/paper/2106.10110/) (2021)
- [Swarm Reinforcement Learning For Adaptive Mesh Refinement](https://mlsanity.com/paper/2304.00818/) (2023)
- [Individual-Level Inverse Reinforcement Learning for Mean Field Games](https://mlsanity.com/paper/2202.06401/) (2022)
- [Online Observer-Based Inverse Reinforcement Learning](https://mlsanity.com/paper/2011.02057/) (2020)
- [Bayesian Inverse Reinforcement Learning for Collective Animal Movement](https://mlsanity.com/paper/2009.04003/) (2020)
- [MisoDICE: Multi-Agent Imitation from Unlabeled Mixed-Quality Demonstrations](https://mlsanity.com/paper/2505.18595/) (2025)
- [Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.10548/) (2023)
- [Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learning](https://mlsanity.com/paper/2310.06801/) (2023)
- [Software Engineering for Collective Cyber-Physical Ecosystems](https://mlsanity.com/paper/2406.04780/) (2024)
- [Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning](https://mlsanity.com/paper/1911.07503/) (2019)
- [Show me what you want: Inverse reinforcement learning to automatically design robot swarms by demonstration](https://mlsanity.com/paper/2301.06864/) (2023)
- [Soft-Bellman Equilibrium in Affine Markov Games: Forward Solutions and Inverse Learning](https://mlsanity.com/paper/2304.00163/) (2023)
- [IRL for Restless Multi-Armed Bandits with Applications in Maternal and Child Health](https://mlsanity.com/paper/2412.08463/) (2024)
- [A model-based framework for learning transparent swarm behaviors](https://mlsanity.com/paper/2103.05343/) (2021)
- [Simulation of robot swarms for learning communication-aware coordination](https://mlsanity.com/paper/2302.13124/) (2023)
- [Generative adversarial imitation learning for robot swarms: Learning from human demonstrations and trained policies](https://mlsanity.com/paper/2603.02783/) (2026)

## Abstract (arXiv)

Inverse reinforcement learning (IRL) has become a useful tool for learning behavioral models from demonstration data. However, IRL remains mostly unexplored for multi-agent systems. In this paper, we show how the principle of IRL can be extended to homogeneous large-scale problems, inspired by the collective swarming behavior of natural systems. In particular, we make the following contributions to the field: 1) We introduce the swarMDP framework, a sub-class of decentralized partially observable Markov decision processes endowed with a swarm characterization. 2) Exploiting the inherent homogeneity of this framework, we reduce the resulting multi-agent IRL problem to a single-agent one by proving that the agent-specific value functions in this model coincide. 3) To solve the corresponding control problem, we propose a novel heterogeneous learning scheme that is particularly tailored to the swarm setting. Results on two example systems demonstrate that our framework is able to produce meaningful local reward models from which we can replicate the observed global system dynamics.