# Beyond Accuracy: A Unified Random Matrix Theory Diagnostic Framework for Crash Classification Models

arXiv: https://arxiv.org/abs/2602.19528 · HTML: https://mlsanity.com/paper/2602.19528/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning](https://mlsanity.com/paper/1810.01075/) (2018)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [Traditional and Heavy-Tailed Self Regularization in Neural Network Models](https://mlsanity.com/paper/1901.08276/) (2019)
- [SETOL: A Semi-Empirical Theory of (Deep) Learning](https://mlsanity.com/paper/2507.17912/) (2025)
- [Accuracy is Not Agreement: Expert-Aligned Evaluation of Crash Narrative Classification Models](https://mlsanity.com/paper/2504.13068/) (2025)
- [Unlocking Insights Addressing Alcohol Inference Mismatch through Database-Narrative Alignment](https://mlsanity.com/paper/2506.19342/) (2025)
- [Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)](https://mlsanity.com/paper/2507.03066/) (2025)

## Abstract (arXiv)

Crash classification models in transportation safety are typically evaluated using accuracy, F1, or AUC, metrics that cannot reveal whether a model is silently overfitting. We introduce a spectral diagnostic framework grounded in Random Matrix Theory (RMT) and Heavy-Tailed Self-Regularization (HTSR) that spans the ML taxonomy: weight matrices for BERT/ALBERT/Qwen2.5, out-of-fold increment matrices for XGBoost/Random Forest, empirical Hessians for Logistic Regression, induced affinity matrices for Decision Trees, and Graph Laplacians for KNN. Evaluating nine model families on two Iowa DOT crash classification tasks (173,512 and 371,062 records respectively), we find that the power-law exponent α provides a structural quality signal: well-regularized models consistently yield α within [2, 4] (mean 2.87 ± 0.34), while overfit variants show α < 2 or spectral collapse. We observe a strong rank correlation between α and expert agreement (Spearman ρ = 0.89, p < 0.001), suggesting spectral quality captures model behaviors aligned with expert reasoning. We propose an α-based early stopping criterion and a spectral model selection protocol, and validate both against cross-validated F1 baselines. Sparse Lanczos approximations make the framework scalable to large datasets.