# Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes

arXiv: https://arxiv.org/abs/2306.04306 · HTML: https://mlsanity.com/paper/2306.04306/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling](https://mlsanity.com/paper/2011.11588/) (2020)
- [Low-resource Languages: A Review of Past Work and Future Challenges](https://mlsanity.com/paper/2006.07264/) (2020)
- [TorchAudio: Building Blocks for Audio and Speech Processing](https://mlsanity.com/paper/2110.15018/) (2021)
- [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [Quantifying Bias in Automatic Speech Recognition](https://mlsanity.com/paper/2103.15122/) (2021)
- [ASR2K: Speech Recognition for Around 2000 Languages without Audio](https://mlsanity.com/paper/2209.02842/) (2022)
- [A Survey of Multilingual Models for Automatic Speech Recognition](https://mlsanity.com/paper/2202.12576/) (2022)
- [AlloVera: A Multilingual Allophone Database](https://mlsanity.com/paper/2004.08031/) (2020)
- [Multilingual acoustic word embedding models for processing zero-resource languages](https://mlsanity.com/paper/2002.02109/) (2020)
- [Feature learning for efficient ASR-free keyword spotting in low-resource languages](https://mlsanity.com/paper/2108.06174/) (2021)

## Built on by (most cited first, 13 of 13)

- [ZIPA: A family of efficient models for multilingual phone recognition](https://mlsanity.com/paper/2505.23170/) (2025)
- [Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision](https://mlsanity.com/paper/2406.02166/) (2024)
- [The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any language](https://mlsanity.com/paper/2311.08323/) (2023)
- [POWSM: A Phonetic Open Whisper-Style Speech Foundation Model](https://mlsanity.com/paper/2510.24992/) (2025)
- [An Empirical Recipe for Universal Phone Recognition](https://mlsanity.com/paper/2603.29042/) (2026)
- [LLM-based phoneme-to-grapheme for phoneme-based speech recognition](https://mlsanity.com/paper/2506.04711/) (2025)
- [HuPER: A Human-Inspired Framework for Phonetic Perception](https://mlsanity.com/paper/2602.01634/) (2026)
- [CUPE: Contextless Universal Phoneme Encoder for Language-Agnostic Speech Processing](https://mlsanity.com/paper/2508.15316/) (2025)
- [Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English](https://mlsanity.com/paper/2508.19270/) (2025)
- [BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encoder](https://mlsanity.com/paper/2606.22824/) (2026)
- [Language-Universal Speech Attributes Modeling for Zero-Shot Multilingual Spoken Keyword Recognition](https://mlsanity.com/paper/2406.02488/) (2024)
- [Phone Segmentation and Recognition through Phonological Activation Mapping](https://mlsanity.com/paper/2607.09020/) (2026)
- [BranchShine-CR: Compact Multilingual IPA Transcription with Self-Conditioned CTC and Consistency Regularization](https://mlsanity.com/paper/2609.29069/) (2026)

## Abstract (arXiv)

This paper proposes Allophant, a multilingual phoneme recognizer. It requires only a phoneme inventory for cross-lingual transfer to a target language, allowing for low-resource recognition. The architecture combines a compositional phone embedding approach with individually supervised phonetic attribute classifiers in a multi-task architecture. We also introduce Allophoible, an extension of the PHOIBLE database. When combined with a distance based mapping approach for grapheme-to-phoneme outputs, it allows us to train on PHOIBLE inventories directly. By training and evaluating on 34 languages, we found that the addition of multi-task learning improves the model's capability of being applied to unseen phonemes and phoneme inventories. On supervised languages we achieve phoneme error rate improvements of 11 percentage points (pp.) compared to a baseline without multi-task learning. Evaluation of zero-shot transfer on 84 languages yielded a decrease in PER of 2.63 pp. over the baseline.