# CrashFormer: A Multimodal Architecture to Predict the Risk of Crash

arXiv: https://arxiv.org/abs/2402.05151 · HTML: https://mlsanity.com/paper/2402.05151/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [RiskOracle: A Minute-level Citywide Traffic Accident Forecasting Framework](https://mlsanity.com/paper/2003.00819/) (2020)
- [Accident Risk Prediction based on Heterogeneous Sparse Data: New Dataset and Insights](https://mlsanity.com/paper/1909.09638/) (2019)
- [A Countrywide Traffic Accident Dataset](https://mlsanity.com/paper/1906.05409/) (2019)
- [Short and Long-term Pattern Discovery Over Large-Scale Geo-Spatiotemporal Data](https://mlsanity.com/paper/1902.06792/) (2019)
- [Real-time Traffic Accident Risk Prediction based on Frequent Pattern Tree](https://mlsanity.com/paper/1701.05691/) (2017)
- [Will there be a construction? Predicting road constructions based on heterogeneous spatiotemporal data](https://mlsanity.com/paper/2209.06813/) (2022)
- [Real-time Travel Time Estimation Using Matrix Factorization](https://mlsanity.com/paper/1912.00455/) (2019)

## Built on by (most cited first, 5 of 5)

- [DetailCLIP: Detail-Oriented CLIP for Fine-Grained Tasks](https://mlsanity.com/paper/2409.06809/) (2024)
- [Masked LoGoNet: Fast and Accurate 3D Image Analysis for Medical Domain](https://mlsanity.com/paper/2402.06190/) (2024)
- [Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happen](https://mlsanity.com/paper/2505.21743/) (2025)
- [Recent Advances in Traffic Accident Analysis and Prediction: A Comprehensive Review of Machine Learning Techniques](https://mlsanity.com/paper/2406.13968/) (2024)
- [ALCo-FM: Adaptive Long-Context Foundation Model for Accident Prediction](https://mlsanity.com/paper/2507.08153/) (2025)

## Abstract (arXiv)

Reducing traffic accidents is a crucial global public safety concern. Accident prediction is key to improving traffic safety, enabling proactive measures to be taken before a crash occurs, and informing safety policies, regulations, and targeted interventions. Despite numerous studies on accident prediction over the past decades, many have limitations in terms of generalizability, reproducibility, or feasibility for practical use due to input data or problem formulation. To address existing shortcomings, we propose CrashFormer, a multi-modal architecture that utilizes comprehensive (but relatively easy to obtain) inputs such as the history of accidents, weather information, map images, and demographic information. The model predicts the future risk of accidents on a reasonably acceptable cadence (i.e., every six hours) for a geographical location of 5.161 square kilometers. CrashFormer is composed of five components: a sequential encoder to utilize historical accidents and weather data, an image encoder to use map imagery data, a raw data encoder to utilize demographic information, a feature fusion module for aggregating the encoded features, and a classifier that accepts the aggregated data and makes predictions accordingly. Results from extensive real-world experiments in 10 major US cities show that CrashFormer outperforms state-of-the-art sequential and non-sequential models by 1.8% in F1-score on average when using ``sparse'' input data.