# Heter-Sim: Heterogeneous multi-agent systems simulation by interactive data-driven optimization

arXiv: https://arxiv.org/abs/1812.00307 · HTML: https://mlsanity.com/paper/1812.00307/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2018.

## Built on by (most cited first, 3 of 3)

- [SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents](https://mlsanity.com/paper/2402.17339/) (2024)
- [Emotional Contagion-Aware Deep Reinforcement Learning for Antagonistic Crowd Simulation](https://mlsanity.com/paper/2105.00854/) (2021)
- [Flow-Opt: Scalable Centralized Multi-Robot Trajectory Optimization with Flow Matching and Differentiable Optimization](https://mlsanity.com/paper/2510.09204/) (2025)

## Abstract (arXiv)

Interactive multi-agent simulation algorithms are used to compute the trajectories and behaviors of different entities in virtual reality scenarios. However, current methods involve considerable parameter tweaking to generate plausible behaviors. We introduce a novel approach (Heter-Sim) that combines physics-based simulation methods with data-driven techniques using an optimization-based formulation. Our approach is general and can simulate heterogeneous agents corresponding to human crowds, traffic, vehicles, or combinations of different agents with varying dynamics. We estimate motion states from real-world datasets that include information about position, velocity, and control direction. Our optimization algorithm considers several constraints, including velocity continuity, collision avoidance, attraction, and direction control. To accelerate the computations, we reduce the search space for both collision avoidance and optimal solution computation. Heter-Sim can simulate tens or hundreds of agents at interactive rates and we compare its accuracy with real-world datasets and prior algorithms. We also perform user studies that evaluate the plausible behaviors generated by our algorithm and a user study that evaluates the plausibility of our algorithm via VR.