mlsanityData to 14 Oct 2026

AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder

arXiv 2501.05332 ↗First posted 9 Jan 2025cs.SD cs.AI eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

This article introduces AnCoGen, a novel method that leverages a masked autoencoder to unify the analysis, control, and generation of speech signals within a single model. AnCoGen can analyze speech by estimating key attributes, such as speaker identity, pitch, content, loudness, signal-to-noise ratio, and clarity index. In addition, it can generate speech from these attributes and allow precise control of the synthesized speech by modifying them. Extensive experiments demonstrated the effectiveness of AnCoGen across speech analysis-resynthesis, pitch estimation, pitch modification, and speech enhancement.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 46 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

26 of 46 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  8. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  9. 2022High Fidelity Neural Audio Compressioncited by 687
  10. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  11. 2018SDR - half-baked or well done?cited by 423
  12. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
14 more
  1. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  2. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  3. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
  4. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
  5. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  6. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 141
  7. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
  8. 2019Phase-aware Speech Enhancement with Deep Complex U-Netcited by 73
  9. 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 68
  10. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  11. 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
  12. 2022Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimationcited by 23
  13. 2022Speech Quality Assessment through MOS using Non-Matching Referencescited by 16
  14. 2023A vector quantized masked autoencoder for audiovisual speech emotion recognitioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Residual Tokens Enhance Masked Autoencoders for Speech Modelingcited by 1
  2. 2026Responsible Benchmarking of Fairness for Automatic Speech Recognitioncited by 1
  3. 2025U-DREAM: Unsupervised Dereverberation guided by a Reverberation Modelcited by 0