Pixel-TTS: Image based Text Rendering for Robust Text-to-Speech
Not on the map. Why?
Abstract (arXiv)
Recent advances in pixel-based text modeling show that representing text as images enables models to exploit visual cues for language understanding. Grounding text in its visual form allows structurally similar characters with different Unicode encodings to produce similar embeddings, benefiting cross-lingual and zero-shot scenarios. Conventional text-based approaches treat each character independently, limiting generalization to unseen characters and requiring embedding expansion during cross-lingual adaptation. We propose Pixel-TTS, a text-to-speech framework for visually grounded speech synthesis. It renders text as images and projects them through a 2D convolutional layer to generate embeddings. This design eliminates embedding matrix expansion during fine-tuning while improving robustness to unseen characters and orthographic variations. Extensive experiments show Pixel-TTS achieves competitive performance with strong baselines, faster convergence and robust zero-shot generalization.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
15 of 21 references.
References · 15
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
- 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
- 2022Language Modelling with Pixelscited by 55
- 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
- 2021Robust Open-Vocabulary Translation from Visual Text Representationscited by 47
Built on by
Not cited yet.