mlsanityData to 14 Oct 2026

When Humans Growl and Birds Speak: High-Fidelity Voice Conversion from Human to Animal and Designed Sounds

arXiv 2505.24336 ↗First posted 30 May 2025eess.AS cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Human to non-human voice conversion (H2NH-VC) transforms human speech into animal or designed vocalizations. Unlike prior studies focused on dog-sounds and 16 or 22.05kHz audio transformation, this work addresses a broader range of non-speech sounds, including natural sounds (lion-roars, birdsongs) and designed voice (synthetic growls). To accomodate generation of diverse non-speech sounds and 44.1kHz high-quality audio transformation, we introduce a preprocessing pipeline and an improved CVAE-based H2NH-VC model, both optimized for human and non-human voices. Experimental results showed that the proposed method outperformed baselines in quality, naturalness, and similarity MOS, achieving effective voice conversion across diverse non-human timbres. Demo samples are available at https://nc-ai.github.io/speech/publications/nonhuman-vc/

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 30 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

26 of 30 references.

References · 26

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  4. 2016Least Squares Generative Adversarial Networkscited by 1,001
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  6. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  7. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  8. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2017VAE with a VampPriorcited by 248
  11. 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
  12. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
14 more
  1. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  2. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  3. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  4. 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
  5. 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
  6. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  7. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptationcited by 30
  8. 2019SPICE: Self-supervised Pitch Estimationcited by 27
  9. 2024T-FOLEY: A Controllable Waveform-Domain Diffusion Model for Temporal-Event-Guided Foley Sound Synthesiscited by 19
  10. 2023PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objectivecited by 18
  11. 2023HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfercited by 9
  12. 2023DDSP-SFX: Acoustically-guided sound effects generation with differentiable digital signal processingcited by 5
  13. 2022Speak Like a Dog: Human to Non-human creature Voice Conversioncited by 3
  14. 2023A Systematic Exploration of Joint-training for Singing Voice Synthesiscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Spectrogram features for audio and speech analysiscited by 0