# Tokenizer Transplantation: Mitigating Autoregressive Collapse in Edge-Efficient Bengali ASR

arXiv: https://arxiv.org/abs/2607.09598 · HTML: https://mlsanity.com/paper/2607.09598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 4 of 4)

- [Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP](https://mlsanity.com/paper/2408.04303/) (2024)
- [OOD-Speech: A Large Bengali Speech Recognition Dataset for Out-of-Distribution Benchmarking](https://mlsanity.com/paper/2305.09688/) (2023)
- [Flavors of Moonshine: Tiny Specialized ASR Models for Edge Devices](https://mlsanity.com/paper/2509.02523/) (2025)
- [Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment](https://mlsanity.com/paper/2602.23070/) (2026)

## Abstract (arXiv)

Lightweight speech recognition models are critical for edge deployment, yet highly optimized architectures like Moonshine often fail on morphologically rich, non-Latin languages such as Bengali. This study identifies the root cause of this failure as the model's English-centric byte-level tokenizer, which fragments Bengali words into high-fertility byte chains and triggers catastrophic autoregressive collapse during inference. To resolve this, a novel vocabulary transplantation pipeline is proposed to replace the decoder vocabulary with the native-script BanglaBERT WordPiece vocabulary and resize the corresponding token embedding matrix. Experimental results demonstrate a reduction in token fertility from 9.16 to 1.30. By decreasing autoregressive sequence length by 85.8%, decoding instability is entirely mitigated. When evaluated on the 882-hour Lipi-Ghor dataset, the modified architecture achieves a competitive 21.54% Word Error Rate (WER) and a Real-Time Factor (RTF) of 0.0053. Ultimately, this research provides a scalable, reproducible blueprint for cross-script adaptation of compact ASR models without the need for resource-intensive pre-training.