mlsanityData to 14 Oct 2026

Pixel-TTS: Image based Text Rendering for Robust Text-to-Speech

arXiv 2606.14750 ↗First posted 5 Jun 2026eess.AS cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in pixel-based text modeling show that representing text as images enables models to exploit visual cues for language understanding. Grounding text in its visual form allows structurally similar characters with different Unicode encodings to produce similar embeddings, benefiting cross-lingual and zero-shot scenarios. Conventional text-based approaches treat each character independently, limiting generalization to unseen characters and requiring embedding expansion during cross-lingual adaptation. We propose Pixel-TTS, a text-to-speech framework for visually grounded speech synthesis. It renders text as images and projects them through a 2D convolutional layer to generate embeddings. This design eliminates embedding matrix expansion during fine-tuning while improving robustness to unseen characters and orthographic variations. Extensive experiments show Pixel-TTS achieves competitive performance with strong baselines, faster convergence and robust zero-shot generalization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

15 of 21 references.

References · 15

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  8. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  9. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  10. 2022Language Modelling with Pixelscited by 55
  11. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  12. 2021Robust Open-Vocabulary Translation from Visual Text Representationscited by 47
3 more
  1. 2025Overcoming Vocabulary Constraints with Pixel-level Fallbackcited by 6
  2. 2025Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignmentcited by 4
  3. 2022Visual onoma-to-wave: environmental sound synthesis from visual onomatopoeias and sound-source imagescited by 3

Built on by

Not cited yet.