# Harnessing Event Sensory Data for Error Pattern Prediction in Vehicles: A Language Model Approach

arXiv: https://arxiv.org/abs/2412.13041 · HTML: https://mlsanity.com/paper/2412.13041/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Neural Temporal Point Processes: A Review](https://mlsanity.com/paper/2104.03528/) (2021)
- [Multi-label Prediction in Time Series Data using Deep Neural Networks](https://mlsanity.com/paper/2001.10098/) (2020)
- [Self-Supervised Contrastive Pre-Training for Multivariate Point Processes](https://mlsanity.com/paper/2402.00987/) (2024)

## Built on by (most cited first, 6 of 6)

- [Towards Practical Multi-label Causal Discovery in High-Dimensional Event Sequences via One-Shot Graph Aggregation](https://mlsanity.com/paper/2509.19112/) (2025)
- [One-Shot Multi-Label Causal Discovery in High-Dimensional Event Sequences](https://mlsanity.com/paper/2509.23213/) (2025)
- [Your Autoregressive Model Already Reveals the Causal Graph](https://mlsanity.com/paper/2602.01135/) (2026)
- [Transforming Vehicle Diagnostics: A Multimodal Approach to Error Patterns Prediction](https://mlsanity.com/paper/2602.01109/) (2026)
- [Multi-Agent Causal Reasoning System for Error Pattern Rule Automation in Vehicles](https://mlsanity.com/paper/2602.01155/) (2026)
- [seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences](https://mlsanity.com/paper/2609.31801/) (2026)

## Abstract (arXiv)

In this paper, we draw an analogy between processing natural languages and processing multivariate event streams from vehicles in order to predict textitwhen and textitwhat error pattern is most likely to occur in the future for a given car. Our approach leverages the temporal dynamics and contextual relationships of our event data from a fleet of cars. Event data is composed of discrete values of error codes as well as continuous values such as time and mileage. Modelled by two causal Transformers, we can anticipate vehicle failures and malfunctions before they happen. Thus, we introduce textitCarFormer, a Transformer model trained via a new self-supervised learning strategy, and textitEPredictor, an autoregressive Transformer decoder model capable of predicting textitwhen and textitwhat error pattern will most likely occur after some error code apparition. Despite the challenges of high cardinality of event types, their unbalanced frequency of appearance and limited labelled data, our experimental results demonstrate the excellent predictive ability of our novel model. Specifically, with sequences of 160 error codes on average, our model is able with only half of the error codes to achieve 80% F1 score for predicting textitwhat error pattern will occur and achieves an average absolute error of 58.4 ± 13.2h textitwhen forecasting the time of occurrence, thus enabling confident predictive maintenance and enhancing vehicle safety.