# Long-term Microscopic Traffic Simulation with History-Masked Multi-agent Imitation Learning

arXiv: https://arxiv.org/abs/2306.06401 · HTML: https://mlsanity.com/paper/2306.06401/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 10 Jun 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [SimNet: Learning Reactive Self-driving Simulations from Real-world Observations](https://mlsanity.com/paper/2105.12332/) (2021)
- [Graph Neural Networks for Modelling Traffic Participant Interaction](https://mlsanity.com/paper/1903.01254/) (2019)
- [Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learning](https://mlsanity.com/paper/1903.05766/) (2019)
- [CCIL: Context-conditioned imitation learning for urban driving](https://mlsanity.com/paper/2305.02649/) (2023)
- [Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation](https://mlsanity.com/paper/2210.03772/) (2022)
- [Exploring the trade off between human driving imitation and safety for traffic simulation](https://mlsanity.com/paper/2208.04803/) (2022)

## Built on by (most cited first, 3 of 3)

- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2403.14879/) (2024)

## Abstract (arXiv)

A realistic long-term microscopic traffic simulator is necessary for understanding how microscopic changes affect traffic patterns at a larger scale. Traditional simulators that model human driving behavior with heuristic rules often fail to achieve accurate simulations due to real-world traffic complexity. To overcome this challenge, researchers have turned to neural networks, which are trained through imitation learning from human driver demonstrations. However, existing learning-based microscopic simulators often fail to generate stable long-term simulations due to the textitcovariate shift issue. To address this, we propose a history-masked multi-agent imitation learning method that removes all vehicles' historical trajectory information and applies perturbation to their current positions during learning. We apply our approach specifically to the urban traffic simulation problem and evaluate it on the real-world large-scale pNEUMA dataset, achieving better short-term microscopic and long-term macroscopic similarity to real-world data than state-of-the-art baselines.