Harnessing Event Sensory Data for Error Pattern Prediction in Vehicles: A Language Model Approach

In this paper, we draw an analogy between processing natural languages and processing multivariate event streams from vehicles in order to predict textitwhen and textitwhat error pattern is most likely to occur in the future for a given car. Our approach leverages the temporal dynamics and contextual relationships of our event data from a fleet of cars. Event data is composed of discrete values of error codes as well as continuous values such as time and mileage. Modelled by two causal Transformers, we can anticipate vehicle failures and malfunctions before they happen. Thus, we introduce textitCarFormer, a Transformer model trained via a new self-supervised learning strategy, and textitEPredictor, an autoregressive Transformer decoder model capable of predicting textitwhen and textitwhat error pattern will most likely occur after some error code apparition. Despite the challenges of high cardinality of event types, their unbalanced frequency of appearance and limited labelled data, our experimental results demonstrate the excellent predictive ability of our novel model. Specifically, with sequences of 160 error codes on average, our model is able with only half of the error codes to achieve 80% F1 score for predicting textitwhat error pattern will occur and achieves an average absolute error of 58.4 ± 13.2h textitwhen forecasting the time of occurrence, thus enabling confident predictive maintenance and enhancing vehicle safety.