# Predicting Intermittent Job Failure Categories for Diagnosis Using Few-Shot Fine-Tuned Language Models

arXiv: https://arxiv.org/abs/2601.22264 · HTML: https://mlsanity.com/paper/2601.22264/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022)
- [Flakify: A Black-Box, Language Model-based Predictor for Flaky Tests](https://mlsanity.com/paper/2112.12331/) (2021)
- [FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repair](https://mlsanity.com/paper/2307.00012/) (2023)
- [Empirical Study of Restarted and Flaky Builds on Travis CI](https://mlsanity.com/paper/2003.11772/) (2020)
- [On the Diagnosis of Flaky Job Failures: Understanding and Prioritizing Failure Categories](https://mlsanity.com/paper/2501.04976/) (2025)
- [Build Optimization: A Systematic Literature Review](https://mlsanity.com/paper/2501.11940/) (2025)
- [On the Illusion of Success: An Empirical Study of Build Reruns and Silent Failures in Industrial CI](https://mlsanity.com/paper/2509.14347/) (2025)
- [FlakeRanker: Automated Identification and Prioritization of Flaky Job Failure Categories](https://mlsanity.com/paper/2503.12312/) (2025)
- [Towards Build Optimization Using Digital Twins](https://mlsanity.com/paper/2503.19381/) (2025)
- [Efficient Detection of Intermittent Job Failures Using Few-Shot Learning](https://mlsanity.com/paper/2507.04173/) (2025)

## Built on by (most cited first, 1 of 1)

- [RETRACE: From Entangled Repair Histories to Reusable Experience for CI Repair](https://mlsanity.com/paper/2610.04658/) (2026)

## Abstract (arXiv)

In principle, Continuous Integration (CI) pipeline failures provide valuable feedback to developers on code-related errors. In practice, however, pipeline jobs often fail intermittently due to non-deterministic tests, network outages, infrastructure failures, resource exhaustion, and other reliability issues. These intermittent (flaky) job failures lead to substantial inefficiencies: wasted computational resources from repeated reruns and significant diagnosis time that distracts developers from core activities and often requires intervention from specialized teams. Prior work has proposed machine learning techniques to detect intermittent failures, but does not address the subsequent diagnosis challenge. To fill this gap, we introduce FlaXifyer, a few-shot learning approach for predicting intermittent job failure categories using pre-trained language models. FlaXifyer requires only job execution logs and achieves 84.3% Macro F1 and 92.0% Top-2 accuracy with just 12 labeled examples per category. We also propose LogSift, an interpretability technique that identifies influential log statements in under one second, reducing review effort by 74.4% while surfacing relevant failure information in 87% of cases. Evaluation on 2,458 job failures from TELUS demonstrates that FlaXifyer and LogSift enable effective automated triage, accelerate failure diagnosis, and pave the way towards the automated resolution of intermittent job failures.