# Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speech

arXiv: https://arxiv.org/abs/2507.13563 · HTML: https://mlsanity.com/paper/2507.13563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 70 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 30 of 32)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus](https://mlsanity.com/paper/2305.18802/) (2023)
- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric](https://mlsanity.com/paper/2004.09584/) (2020)
- [A Survey on Speech Large Language Models for Understanding](https://mlsanity.com/paper/2410.18908/) (2024)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra](https://mlsanity.com/paper/2305.13686/) (2023)
- [WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark](https://mlsanity.com/paper/2406.05763/) (2024)
- [MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation](https://mlsanity.com/paper/2312.11825/) (2023)
- [Deep Learning Based Assessment of Synthetic Speech Naturalness](https://mlsanity.com/paper/2104.11673/) (2021)
- [Transformer based Grapheme-to-Phoneme Conversion](https://mlsanity.com/paper/2004.06338/) (2020)
- [DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement](https://mlsanity.com/paper/2305.08227/) (2023)
- [VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Design](https://mlsanity.com/paper/2307.16430/) (2023)
- [Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation](https://mlsanity.com/paper/2501.15907/) (2025)
- [Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning](https://mlsanity.com/paper/2506.17525/) (2025)
- [TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touch](https://mlsanity.com/paper/2412.08237/) (2024)
- [Golos: Russian Dataset for Speech Research](https://mlsanity.com/paper/2106.10161/) (2021)
- [Large Language User Interfaces: Voice Interactive User Interfaces powered by LLMs](https://mlsanity.com/paper/2402.07938/) (2024)
- [RUSLAN: Russian Spoken Language Corpus for Speech Synthesis](https://mlsanity.com/paper/1906.11645/) (2019)
- [Prosody Analysis of Audiobooks](https://mlsanity.com/paper/2310.06930/) (2023)
- [Breaking the Transcription Bottleneck: Fine-tuning ASR Models for Extremely Low-Resource Fieldwork Languages](https://mlsanity.com/paper/2506.17459/) (2025)
- [VoiceRestore: Flow-Matching Transformers for Speech Recording Quality Restoration](https://mlsanity.com/paper/2501.00794/) (2025)
- [TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data](https://mlsanity.com/paper/2506.15614/) (2025)

## Abstract (arXiv)

We introduce Balalaika, an open-source, data-centric pipeline for processing audio and producing prosody-aware annotations. It combines semantic VAD for context-preserving segmentation, multi-ASR ensembling with ROVER consensus decoding, while retaining optional word-level timestamps, followed by automatic quality and speaker-purity filtering. The text is further enriched with punctuation restoration, lexical stress and "textipae/textipaHe" normalization, and IPA phonemes. Using Balalaika, we build a 5.1k-hour multi-source Russian corpus with rich annotations, and show consistent gains under equalized training budgets for both speech denoising and TTS; ablations confirm complementary benefits of stress and punctuation and improved synthesis with stricter MOS filtering. The datasets are publicly available at hrefhttps://huggingface.co/collections/lab260/balalaika-datasetunderlineHuggingFace