# XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts

arXiv: https://arxiv.org/abs/2609.24770 · HTML: https://mlsanity.com/paper/2609.24770/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models](https://mlsanity.com/paper/2403.03100/) (2024)
- [VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.05370/) (2024)
- [The VoiceMOS Challenge 2022](https://mlsanity.com/paper/2203.11389/) (2022)
- [Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment](https://mlsanity.com/paper/2406.17957/) (2024)
- [How do Voices from Past Speech Synthesis Challenges Compare Today?](https://mlsanity.com/paper/2105.02373/) (2021)
- [The AudioMOS Challenge 2025](https://mlsanity.com/paper/2509.01336/) (2025)
- [Towards Frame-level Quality Predictions of Synthetic Speech](https://mlsanity.com/paper/2508.10374/) (2025)
- [Phoneme Discretized Saliency Maps for Explainable Detection of AI-Generated Voice](https://mlsanity.com/paper/2406.10422/) (2024)
- [Speech Quality-Based Localization of Low-Quality Speech and Text-to-Speech Synthesis Artefacts](https://mlsanity.com/paper/2601.21886/) (2026)

## Abstract (arXiv)

Localising artifacts in synthetic speech remains challenging, as most evaluation methods yield only global quality scores. This paper presents XSQ-AST, a framework that combines the SQ-AST speech quality model with WhisperX phoneme alignment and multiple saliency methods to produce temporally localised artifact diagnostics without model retraining. Saliency maps are projected onto continuous distributions via kernel density estimation and onto phoneme boundaries via phoneme-discretised saliency maps. A 40-participant listening test validated the framework across five perceptual dimensions. Attention Rollout, Attention Flow and an adapted GradCAM produced temporal distributions that correlated with listener highlights, with different methods best suited to different artifact types. An AUC-ROC analysis confirmed discrimination above chance.