# TASLA: Text-Aligned Speech Tokens with Multiple Layer-Aggregation

arXiv: https://arxiv.org/abs/2510.14934 · HTML: https://mlsanity.com/paper/2510.14934/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2025.

## Built on by (most cited first, 3 of 3)

- [Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization](https://mlsanity.com/paper/2601.23174/) (2026)
- [TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling](https://mlsanity.com/paper/2603.12350/) (2026)
- [StreamAlign: Streaming Text-Aligned Speech Tokenization](https://mlsanity.com/paper/2609.09719/) (2026)

## Abstract (arXiv)

We propose Text-Aligned Speech Tokens with Multiple Layer-Aggregation (TASLA), which is a text-aligned speech tokenization framework that aims to address the problem that under a low-frame-rate and text-aligned regime, single-source speech tokens may lose acoustic details during reconstruction. On the other hand, this paper further explains how different encoder layers collaborate to capture comprehensive acoustic features for tokenization. Previous work, TASTE, proposed the text-aligned speech tokenization framework, which is a LM-friendly architecture, but struggles to capture acoustic details. We address this trade-off with two components: Multi-Layer Dynamic Attention (MLDA), which lets each text position adaptively mix shallow/deep features from a frozen speech encoder, and Finite Scalar Quantization (FSQ), a simple per-dimension discretization with smooth optimization. At about 2.62 Hz (tokens/s), TASLA consistently improves prosody and achieves competitive quality over TASTE on in-domain (LibriSpeech) and OOD (EXPRESSO, Voxceleb) sets. We further demonstrate that dynamic layer mixing is correlated with spectral flux and explains why MLDA preserves prosody under a low frame rate with extreme feature compression.