# A Language Modeling Approach to Diacritic-Free Hebrew TTS

arXiv: https://arxiv.org/abs/2407.12206 · HTML: https://mlsanity.com/paper/2407.12206/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [AudioGen: Textually Guided Audio Generation](https://mlsanity.com/paper/2209.15352/) (2022)
- [Speech Resynthesis from Discrete Disentangled Self-Supervised Representations](https://mlsanity.com/paper/2104.00355/) (2021)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision](https://mlsanity.com/paper/2302.03540/) (2023)
- [Text-Free Prosody-Aware Generative Spoken Language Modeling](https://mlsanity.com/paper/2109.03264/) (2021)
- [I Hear Your True Colors: Image Guided Audio Generation](https://mlsanity.com/paper/2211.03089/) (2022)
- [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023)
- [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https://mlsanity.com/paper/2402.01912/) (2024)
- [AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With](https://mlsanity.com/paper/2104.04052/) (2021)
- [fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit](https://mlsanity.com/paper/2109.06912/) (2021)
- [From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion](https://mlsanity.com/paper/2308.02560/) (2023)
- [OverFlow: Putting flows on top of neural transducers for better TTS](https://mlsanity.com/paper/2211.06892/) (2022)
- [ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development](https://mlsanity.com/paper/2307.08720/) (2023)
- [HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing](https://mlsanity.com/paper/2407.07566/) (2024)

## Built on by (most cited first, 2 of 2)

- [Scaling Analysis of Interleaved Speech-Text Language Models](https://mlsanity.com/paper/2504.02398/) (2025)
- [Phonikud: Overcoming Phonetic Underspecification for Hebrew Text-To-Speech](https://mlsanity.com/paper/2506.12311/) (2025)

## Abstract (arXiv)

We tackle the task of text-to-speech (TTS) in Hebrew. Traditional Hebrew contains Diacritics, which dictate the way individuals should pronounce given words, however, modern Hebrew rarely uses them. The lack of diacritics in modern Hebrew results in readers expected to conclude the correct pronunciation and understand which phonemes to use based on the context. This imposes a fundamental challenge on TTS systems to accurately map between text-to-speech. In this work, we propose to adopt a language modeling Diacritics-Free approach, for the task of Hebrew TTS. The model operates on discrete speech representations and is conditioned on a word-piece tokenizer. We optimize the proposed method using in-the-wild weakly supervised data and compare it to several diacritic-based TTS systems. Results suggest the proposed method is superior to the evaluated baselines considering both content preservation and naturalness of the generated speech. Samples can be found under the following link: pages.cs.huji.ac.il/adiyoss-lab/HebTTS/