Vagdhenu: A Vrutta (Meter) Aware Shloka-to-Chant (TTS) System for Sanskrit
Not on the map. Why?
Abstract (arXiv)
We present Vagdhenu, a vrutta (meter) aware shloka-to-chant system for Sanskrit: a text-to-speech system that maps a metrical verse to its chanted parayana recitation at high fidelity. This is an experience report, not a new architecture. We take an off-the-shelf flow-matching TTS backbone and a large-scale neural vocoder, and add the components a faithful Sanskrit chant pipeline needs: a frontend that routes Sanskrit through Kannada orthography to avoid the Hindi-style schwa deletion that Devanagari triggers in Indic models; a frontend that obeys subtle Sanskrit phonology (visarga sandhi with its jihvamuliya and upadhmaniya allophones, the aspiration contrast of alpaprana and mahaprana, and the dental, retroflex, and palatal sibilants kept distinct); and a vrutta-aware mechanism that detects the meter and picks an exactly matched reference under a half-reference rule. We report a negative result that shaped the system: in a self-infilling flow-matching backbone, a text-side prosody conditioner is architecturally inert, because the model recovers pitch from the context mel and the embedding gets no gradient; the reference clip and a voice-steering retrain are the only working prosody levers. We also report a comparative lineage across four families (StyleTTS2, VITS2, Matcha-TTS, and the flow-matching backbone), where each earlier family hit a ceiling on conjuncts or prosody that a five-hour clone cleared at an expert MOS near 4.6. The system shipped two deployments: a 32-chapter, 5183-verse video corpus (about 17.5 hours) and an audio app covering about 18000 verses across 12 books. We release the frontend, inference and training code, weights, a single-speaker chant dataset, and an interactive demo.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
16 of 22 references.
References · 16
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
- 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
4 more
- 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
- 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
- 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 67
- 2023VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Designcited by 25
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Hierarchical Generative Modeling of Melodic Vocal Contours in Hindustani Classical Music
- 2026Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer
- 2026VocalParse: Towards Unified and Scalable Singing Voice Transcription with Large Audio Language Models
- 2026Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speech
- 2026MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis
- 2025InstructAudio: Unified speech and music generation with natural language instruction
- 2026UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions
- 2026Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper