# When Less Is More? Diagnosing ASR Predictions in Sardinian via Layer-Wise Decoding

arXiv: https://arxiv.org/abs/2602.10350 · HTML: https://mlsanity.com/paper/2602.10350/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 6 of 6)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Jump to Conclusions: Short-Cutting Transformers With Linear Transformations](https://mlsanity.com/paper/2303.09435/) (2023)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers](https://mlsanity.com/paper/2310.03686/) (2023)
- [Languages in Whisper-Style Speech Encoders Align Both Phonetically and Semantically](https://mlsanity.com/paper/2505.19606/) (2025)

## Abstract (arXiv)

Recent studies have shown that intermediate layers in multilingual speech models often encode more phonetically accurate representations than the final output layer. In this work, we apply a layer-wise decoding strategy to a pretrained Wav2Vec2 model to investigate how phoneme-level predictions evolve across encoder layers, focusing on Campidanese Sardinian, a low-resource language. We show that truncating upper transformer layers leads to improved Phoneme Error Rates (PER), with the best performance achieved not at the final layer, but two layers earlier. Through fine-grained alignment analysis, we find that intermediate predictions better preserve segmental identity, avoid overgeneration, and reduce certain classes of phonological errors. We also introduce the notion of regressive errors, cases where correct predictions at intermediate layers are overwritten by errors at the final layer. These regressions highlight the limitations of surface-level error metrics and reveal how deeper layers may generalize or abstract away from acoustic detail. Our findings support the use of early-layer probing as a diagnostic tool for ASR models, particularly in low-resource settings where standard evaluation metrics may fail to capture linguistically meaningful behavior.