# Bayesian Inverse Reinforcement Learning for Collective Animal Movement

arXiv: https://arxiv.org/abs/2009.04003 · HTML: https://mlsanity.com/paper/2009.04003/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 8 Sep 2020.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 14 of 14)

- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Hamiltonian Monte Carlo for Hierarchical Models](https://mlsanity.com/paper/1312.0906/) (2013)
- [Automatic Variational Inference in Stan](https://mlsanity.com/paper/1506.03431/) (2015)
- [Inverse Reinforcement Learning in Swarm Systems](https://mlsanity.com/paper/1602.05450/) (2016)
- [Inverse Reinforcement Learning via Deep Gaussian Process](https://mlsanity.com/paper/1512.08065/) (2015)
- [Inverse Reinforcement Learning from Summary Data](https://mlsanity.com/paper/1703.09700/) (2017)
- [A Bayesian Approach to Policy Recognition and State Representation Learning](https://mlsanity.com/paper/1605.01278/) (2016)
- [Dynamic social networks based on movement](https://mlsanity.com/paper/1512.07607/) (2015)
- [Process convolution approaches for modeling interacting trajectories](https://mlsanity.com/paper/1703.02112/) (2017)
- [Modelling collective motion based on the principle of agency](https://mlsanity.com/paper/1712.01334/) (2017)
- [Hierarchical Subtask Discovery With Non-Negative Matrix Factorization](https://mlsanity.com/paper/1708.00463/) (2017)
- [Animal Movement Models with Mechanistic Selection Functions](https://mlsanity.com/paper/1911.03549/) (2019)

## Built on by (most cited first, 3 of 3)

- [Statistical Deep Learning for Spatial and Spatio-Temporal Data](https://mlsanity.com/paper/2206.02218/) (2022)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Spherical Poisson Point Process Intensity Function Modeling and Estimation with Measure Transport](https://mlsanity.com/paper/2201.09485/) (2022)

## Abstract (arXiv)

Agent-based methods allow for defining simple rules that generate complex group behaviors. The governing rules of such models are typically set a priori and parameters are tuned from observed behavior trajectories. Instead of making simplifying assumptions across all anticipated scenarios, inverse reinforcement learning provides inference on the short-term (local) rules governing long term behavior policies by using properties of a Markov decision process. We use the computationally efficient linearly-solvable Markov decision process to learn the local rules governing collective movement for a simulation of the self propelled-particle (SPP) model and a data application for a captive guppy population. The estimation of the behavioral decision costs is done in a Bayesian framework with basis function smoothing. We recover the true costs in the SPP simulation and find the guppies value collective movement more than targeted movement toward shelter.