mlsanityData to 7 Oct 2026

Phone Segmentation and Recognition through Phonological Activation Mapping

arXiv 2607.09020 ↗First posted 10 Jul 2026eess.AS cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce two simple but effective lightweight, gradient-descent-free prediction heads: a recognition head and a segmentation head. Our method requires less than a minute of phonetic transcriptions, and generalizes to unseen phones during training. Across a diverse range of datasets, our approach attains strong segmentation and recognition performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 61 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

34 of 61 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  6. 2015Attention-Based Models for Speech Recognitioncited by 400
  7. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  8. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  9. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  10. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 70
  11. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  12. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
22 more
  1. 2021Phone-to-audio alignment without text: A Semi-supervised Approachcited by 25
  2. 2020Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networkscited by 24
  3. 2022Opening the Black Box of wav2vec Feature Encodercited by 18
  4. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 18
  5. 2025Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessmentcited by 15
  6. 2021Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Codingcited by 14
  7. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  8. 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
  9. 2022Phoneme Segmentation Using Self-Supervised Speech Modelscited by 11
  10. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 11
  11. 2023Speech Intelligibility Assessment of Dysarthric Speech by using Goodness of Pronunciation with Uncertainty Quantificationcited by 10
  12. 2024Phonetic Segmentation of the UCLA Phonetics Lab Archivecited by 9
  13. 2024Wav2Gloss: Generating Interlinear Glossed Text from Speechcited by 8
  14. 2026Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspacescited by 7
  15. 2021Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experimentscited by 5
  16. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  17. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  18. 2026Prosodic ABX: A Language-Agnostic Method for Measuring Prosodic Contrast in Speech Representationscited by 3
  19. 2025What do Speech Foundation Models Learn? Analysis and Applicationscited by 2
  20. 2024A Simple HMM with Self-Supervised Representations for Phone Segmentationcited by 1
  21. 2026Scaling Human and G2P Supervision for Robust Phonetic Transcriptioncited by 1
  22. 2026Speech Playground: An Interactive Tool for Speech Analysis and Comparisoncited by 1

Built on by

Not cited yet.