mlsanityData to 7 Oct 2026

When More Modalities Hurt: Modality Dropout for Heavy-Duty Vehicle Engine Diagnostics

arXiv 2608.23161 ↗First posted 24 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Heavy-duty vehicle diagnostics generate three disconnected data modalities: unstructured multi- lingual service complaints, high-dimensional sensor telemetry with over 80% missing values, and Diagnostic Trouble Codes (DTCs). We investigate whether fusing these modalities improves engine component classification on a proprietary dataset from a major truck manufacturer. Through 5-fold cross-validation across multiple model configurations spanning three model families on five engine component classes (885 samples, the full cross-database matched population for this manufacturer), we find that naive fusion provides modest gains over text alone (65.3%). However, modality dropout during training, which randomly disables entire modalities per batch, forces the network to exploit weaker inputs and achieves 68.8% accuracy on text+DTC fusion (weighted F1: 0.67), a 3.5-point improvement over text-only (65.3%, weighted F1: 0.64) and the best result across all methods including logistic regression and gradient-boosted trees. Per-class analysis shows that the dominant modality varies by fault type: text describes symptoms, DTCs encode structured fault signals, and sensors measure physical state. On intake/exhaust faults, sensors alone reach 93% where text achieves 80%. On fuel system faults, fusion with modality dropout nearly triples accuracy from 15% to 38% over text alone. To our knowledge, this is the first application of three-way modality fusion combining text, sensors, and fault codes in industrial vehicle diagnostics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 16 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

8 of 16 references.

References · 8

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  4. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  5. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  6. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  7. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation Systemcited by 138
  8. 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 35

Built on by

Not cited yet.