# HuPER: A Human-Inspired Framework for Phonetic Perception

arXiv: https://arxiv.org/abs/2602.01634 · HTML: https://mlsanity.com/paper/2602.01634/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 72 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 29 of 29)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head](https://mlsanity.com/paper/2304.12995/) (2023)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [Self-Supervised Speech Representations are More Phonetic than Semantic](https://mlsanity.com/paper/2406.08619/) (2024)
- [speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment](https://mlsanity.com/paper/2104.01378/) (2021)
- [ByT5 model for massively multilingual grapheme-to-phoneme conversion](https://mlsanity.com/paper/2204.03067/) (2022)
- [Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages](https://mlsanity.com/paper/2511.09690/) (2025)
- [BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models](https://mlsanity.com/paper/2306.01506/) (2023)
- [WavJourney: Compositional Audio Creation with Large Language Models](https://mlsanity.com/paper/2307.14335/) (2023)
- [ZIPA: A family of efficient models for multilingual phone recognition](https://mlsanity.com/paper/2505.23170/) (2025)
- [POWSM: A Phonetic Open Whisper-Style Speech Foundation Model](https://mlsanity.com/paper/2510.24992/) (2025)
- [Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes](https://mlsanity.com/paper/2306.04306/) (2023)
- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [Phonetic Segmentation of the UCLA Phonetics Lab Archive](https://mlsanity.com/paper/2403.19509/) (2024)
- [Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization](https://mlsanity.com/paper/2506.13331/) (2025)
- [MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios](https://mlsanity.com/paper/2308.12490/) (2023)
- [Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection](https://mlsanity.com/paper/2505.16351/) (2025)
- [Speech World Model: Causal State-Action Planning with Explicit Reasoning for Speech](https://mlsanity.com/paper/2512.05933/) (2025)

## Built on by (most cited first, 3 of 3)

- [An Empirical Recipe for Universal Phone Recognition](https://mlsanity.com/paper/2603.29042/) (2026)
- [Scaling Human and G2P Supervision for Robust Phonetic Transcription](https://mlsanity.com/paper/2606.16019/) (2026)
- [LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features](https://mlsanity.com/paper/2606.28445/) (2026)

## Abstract (arXiv)

We propose HuPER, a human-inspired framework that models phonetic perception as adaptive inference over acoustic-phonetics evidence and linguistic knowledge. With only 100 hours of training data, HuPER achieves state-of-the-art phonetic error rates on five English benchmarks and strong zero-shot transfer to 95 unseen languages. HuPER is also the first framework to enable adaptive, multi-path phonetic perception under diverse acoustic conditions. All training data, models, and code are open-sourced. Code and demo avaliable at https://github.com/Berkeley-Speech-Group/HuPER.