# NAST: Noise Aware Speech Tokenization for Speech Language Models

arXiv: https://arxiv.org/abs/2406.11037 · HTML: https://mlsanity.com/paper/2406.11037/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2024.

## Built on by (most cited first, 13 of 13)

- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [Salmon: A Suite for Acoustic Language Model Evaluation](https://mlsanity.com/paper/2409.07437/) (2024)
- [Sylber: Syllabic Embedding Representation of Speech from Raw Audio](https://mlsanity.com/paper/2410.07168/) (2024)
- [FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks](https://mlsanity.com/paper/2502.04465/) (2025)
- [LAST: Language Model Aware Speech Tokenization](https://mlsanity.com/paper/2409.03701/) (2024)
- [DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models](https://mlsanity.com/paper/2410.24177/) (2024)
- [SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervision](https://mlsanity.com/paper/2512.20308/) (2025)
- [PAST: Phonetic-Acoustic Speech Tokenizer](https://mlsanity.com/paper/2505.14470/) (2025)
- [DM-Codec: Distilling Multimodal Representations for Speech Tokenization](https://mlsanity.com/paper/2410.15017/) (2024)
- [StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs](https://mlsanity.com/paper/2509.22220/) (2025)
- [Frontend Token Enhancement for Token-Based Speech Recognition](https://mlsanity.com/paper/2602.04217/) (2026)
- [Content is What Remains: Invariant Speech Tokenization from Parallel Utterances](https://mlsanity.com/paper/2607.19033/) (2026)

## Abstract (arXiv)

Speech tokenization is the task of representing speech signals as a sequence of discrete units. Such representations can be later used for various downstream tasks including automatic speech recognition, text-to-speech, etc. More relevant to this study, such representation serves as the basis of Speech Language Models. In this work, we tackle the task of speech tokenization under the noisy setup and present NAST: Noise Aware Speech Tokenization for Speech Language Models. NAST is composed of three main components: (i) a predictor; (ii) a residual encoder; and (iii) a decoder. We evaluate the efficiency of NAST considering several spoken language modeling tasks and show that NAST is superior to the evaluated baselines across all setups. Lastly, we analyze NAST and show its disentanglement properties and robustness to signal variations in the form of noise, reverberation, pitch-shift, and time-stretch. Code and pre-trained models are available at https://github.com/ShovalMessica/NAST.