mlsanityData to 20 Oct 2026

Evaluating Pre-trained Speech Encoders for Spontaneous Speech Detection and Out of Domain Synthetic Speech Generalisation in Indic Languages

arXiv 2608.12536 ↗First posted 12 Aug 2026eess.AScited by 0

On the map: Speech and audio language models → Anti-spoofing and adversarial audio attacks

Abstract (arXiv)

Transformer-based models have shown strong accuracy in distinguishing spontaneous from scripted speech and natural from synthetic speech, but these results are established on a narrow set of well-resourced language benchmarks and have not been extended across Indic languages, nor has embedding geometry been used to explain encoder behaviour or deepfake generalisation failure. We address these gaps by evaluating five frozen transformer encoders, AST, Vaani-FastConformer, Wav2vec2, Whisper and BEATs, across 22 Indic languages, and by conducting a multi-system TTS generalisation experiment across four TTS models. Beyond accuracy, we present language isolation probing and centroid proximity analysis. Probing reveals an encoder-dependent trade-off between language-discriminability and spontaneity detection. Centroid analysis shows that out-of-domain generalisation is predicted by a training system's proximity to unseen TTS embeddings, not its distance from natural speech, a finding with direct implications for training data selection in real-world deepfake detectors.

Key facts. No citing papers in the map yet (data through 20 Oct 2026). Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models. Builds on 14 of its 28 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

14 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Where are we in audio deepfake detection? A systematic analysis over generative and detection models6 Oct

Other references · 13

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,671
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,388
  3. 2021AST: Audio Spectrogram Transformercited by 678
  4. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 422
  5. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 239
  6. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 235
  7. 2020End-to-end anti-spoofing with RawNet2cited by 196
  8. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 106
  9. 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 49
  10. 2021ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plancited by 22
  11. 2026VAANI: Capturing the language landscape for an inclusive digital Indiacited by 14
  12. 2024Classification of Spontaneous and Scripted Speech for Multilingual Audiocited by 5
1 more
  1. 2017Learning Spontaneity to Improve Emotion Recognition In Speechcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Indic-CodecFake meets SATYAM: Towards Detecting Neural Audio Codec Synthesized Speech Deepfakes in Indic Languages
  2. 2026A Comparative Study of Pre-trained Speech Encoders and Training Objectives for Large-Scale Indic Spoken Language Identification
  3. 2026Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages
  4. 2026Audio Deepfake Detection in the Age of Advanced Text-to-Speech models
  5. 2025ShiftySpeech: A Large-Scale Synthetic Speech Dataset with Distribution Shifts
  6. 2024Spoken Language Corpora Augmentation with Domain-Specific Voice-Cloned Speech
  7. 2026On the Interpretability of Whisper Encodings Using Sparse Autoencoders
  8. 2025Multilingual Dataset Integration Strategies for Robust Audio Deepfake Detection: A SAFE Challenge System