# Swarm Behavior Cloning

arXiv: https://arxiv.org/abs/2412.07617 · HTML: https://mlsanity.com/paper/2412.07617/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 17 of 17)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [Behavior Transformers: Cloning k modes with one stone](https://mlsanity.com/paper/2206.11251/) (2022)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [Recent Advances in Imitation Learning from Observation](https://mlsanity.com/paper/1905.13566/) (2019)
- [Fighting Copycat Agents in Behavioral Cloning from Observation Histories](https://mlsanity.com/paper/2010.14876/) (2020)
- [Imitation Learning: Progress, Taxonomies and Challenges](https://mlsanity.com/paper/2106.12177/) (2021)
- [Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble](https://mlsanity.com/paper/2205.09284/) (2022)
- [Attention-Based Recurrence for Multi-Agent Reinforcement Learning under Stochastic Partial Observability](https://mlsanity.com/paper/2301.01649/) (2023)
- [A Strong Baseline for Batch Imitation Learning](https://mlsanity.com/paper/2302.02788/) (2023)
- [Robust Maximum Entropy Behavior Cloning](https://mlsanity.com/paper/2101.01251/) (2021)
- [Case-Based Inverse Reinforcement Learning Using Temporal Coherence](https://mlsanity.com/paper/2206.05827/) (2022)

## Abstract (arXiv)

In sequential decision-making environments, the primary approaches for training agents are Reinforcement Learning (RL) and Imitation Learning (IL). Unlike RL, which relies on modeling a reward function, IL leverages expert demonstrations, where an expert policy πe (e.g., a human) provides the desired behavior. Formally, a dataset D of state-action pairs is provided: D = (s, a = πe(s)). A common technique within IL is Behavior Cloning (BC), where a policy π(s) = a is learned through supervised learning on D. Further improvements can be achieved by using an ensemble of N individually trained BC policies, denoted as E = πi(s)1 ≤ i ≤ N. The ensemble's action a for a given state s is the aggregated output of the N actions: a = frac1N sumi πi(s). This paper addresses the issue of increasing action differences -- the observation that discrepancies between the N predicted actions grow in states that are underrepresented in the training data. Large action differences can result in suboptimal aggregated actions. To address this, we propose a method that fosters greater alignment among the policies while preserving the diversity of their computations. This approach reduces action differences and ensures that the ensemble retains its inherent strengths, such as robustness and varied decision-making. We evaluate our approach across eight diverse environments, demonstrating a notable decrease in action differences and significant improvements in overall performance, as measured by mean episode returns.