mlsanityData to 7 Oct 2026

POWSM: A Phonetic Open Whisper-Style Speech Foundation Model

arXiv 2510.24992 ↗First posted 28 Oct 2025cs.CL eess.AScited by 14

Not on the map. Why?

Abstract (arXiv)

Recent advances in spoken language processing have led to substantial progress in phonetic tasks such as automatic speech recognition (ASR), phone recognition (PR), grapheme-to-phoneme conversion (G2P), and phoneme-to-grapheme conversion (P2G). Despite their conceptual similarity, these tasks have largely been studied in isolation, each relying on task-specific architectures and datasets. In this paper, we introduce POWSM (Phonetic Open Whisper-style Speech Model), the first unified framework capable of jointly performing multiple phone-related tasks. POWSM enables seamless conversion between audio, text (graphemes), and phones, opening up new possibilities for universal and low-resource speech processing. Our model outperforms or matches specialized PR models of similar size (Wav2Vec2Phoneme and ZIPA) while jointly supporting G2P, P2G, and ASR. Our training data, code and models are released to foster open science.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 30 of its 72 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

30 of 72 references.

References · 30

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  6. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  7. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  8. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  9. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  10. 2023Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggerscited by 72
  11. 2021speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessmentcited by 57
  12. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
18 more
  1. 2023Can Whisper perform speech-based in-context learning?cited by 35
  2. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 31
  3. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
  4. 2020Phonotactic Complexity and its Trade-offscited by 23
  5. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 18
  6. 2024Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervisioncited by 17
  7. 2025OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Modelscited by 17
  8. 2023Universal Automatic Phonetic Transcription into the International Phonetic Alphabetcited by 16
  9. 2025Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessmentcited by 15
  10. 2020A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baselinecited by 14
  11. 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
  12. 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
  13. 2022Subword Dictionary Learning and Segmentation Techniques for Automatic Speech Recognition in Tamil and Kannadacited by 10
  14. 2024Wav2Gloss: Generating Interlinear Glossed Text from Speechcited by 8
  15. 2025In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varietiescited by 7
  16. 2021Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experimentscited by 5
  17. 2024PolyIPA -- Multilingual Phoneme-to-Grapheme Conversion Modelcited by 4
  18. 2025Speech-to-Text Translation with Phoneme-Augmented CoT: Enhancing Cross-Lingual Transfer in Low-Resource Scenarioscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

  1. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  2. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  3. 2026Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modelingcited by 4
  4. 2026HuPER: A Human-Inspired Framework for Phonetic Perceptioncited by 3
  5. 2026OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Modelscited by 1
  6. 2026Scaling Human and G2P Supervision for Robust Phonetic Transcriptioncited by 1
  7. 2026ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversioncited by 1
  8. 2026BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encodercited by 1
  9. 2026Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliterationcited by 0
  10. 2026Phone Segmentation and Recognition through Phonological Activation Mappingcited by 0
  11. 2026Quantifying the Generation Modality Gap in Speech-Text Language Modelscited by 0
  12. 2026BranchShine-CR: Compact Multilingual IPA Transcription with Self-Conditioned CTC and Consistency Regularizationcited by 0
2 more
  1. 2026Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoringcited by 0
  2. 2026SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabiccited by 0