mlsanityData to 7 Oct 2026

PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS

arXiv 2302.12391 ↗First posted 24 Feb 2023eess.AS cs.LG cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Previous pitch-controllable text-to-speech (TTS) models rely on directly modeling fundamental frequency, leading to low variance in synthesized speech. To address this issue, we propose PITS, an end-to-end pitch-controllable TTS model that utilizes variational inference to model pitch. Based on VITS, PITS incorporates the Yingram encoder, the Yingram decoder, and adversarial training of pitch-shifted synthesis to achieve pitch-controllability. Experiments demonstrate that PITS generates high-quality speech that is indistinguishable from ground truth speech and has high pitch-controllability without quality degradation. Code, audio samples, and demo are available at https://github.com/anonymous-pits/pits.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

30 of 41 references.

References · 30

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2016Least Squares Generative Adversarial Networkscited by 1,000
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  11. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  12. 2019Residual Flows for Invertible Generative Modelingcited by 256
18 more
  1. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  2. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  3. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  4. 2020End-to-End Adversarial Text-to-Speechcited by 77
  5. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  6. 2022Pushing the limits of raw waveform speaker recognitioncited by 49
  7. 2021One TTS Alignment To Rule Them Allcited by 47
  8. 2021VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesiscited by 38
  9. 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
  10. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
  11. 2022Avocodo: Generative Adversarial Network for Artifact-free Vocodercited by 18
  12. 2022WavThruVec: Latent speech representation as intermediate features for neural speech synthesiscited by 17
  13. 2021FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesiscited by 12
  14. 2021Assem-VC: Realistic Voice Conversion by Assembling Modern Speech Synthesis Techniquescited by 11
  15. 2022PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mappingcited by 8
  16. 2022Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesiscited by 7
  17. 2022Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesiscited by 6
  18. 2019GCI detection from raw speech using a fully-convolutional networkcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 15
  2. 2023VIFS: An End-to-End Variational Inference for Foley Sound Synthesiscited by 2
  3. 2026Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codeccited by 1
  4. 2024Super Monotonic Alignment Searchcited by 0