mlsanityData to 7 Oct 2026

S2Cap: A Benchmark and a Baseline for Singing Style Captioning

arXiv 2409.09866 ↗First posted 15 Sep 2024cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Singing voices contain much richer information than common voices, including varied vocal and acoustic properties. However, current open-source audio-text datasets for singing voices capture only a narrow range of attributes and lack acoustic features, leading to limited utility towards downstream tasks, such as style captioning. To fill this gap, we formally define the singing style captioning task and present S2Cap, a dataset of singing voices with detailed descriptions covering diverse vocal, acoustic, and demographic characteristics. Using this dataset, we develop an efficient and straightforward baseline algorithm for singing style captioning. The dataset is available at https://zenodo.org/records/15673764.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 15 Sep 2024.

Builds on

40 of 68 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  12. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
28 more
  1. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  2. 2021AST: Audio Spectrogram Transformercited by 634
  3. 2019Clotho: An Audio Captioning Datasetcited by 354
  4. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  5. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  6. 2016Movie Descriptioncited by 180
  7. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  8. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
  9. 2022Hybrid Transformers for Music Source Separationcited by 134
  10. 2016Improved Image Captioning via Policy Gradient optimization of SPIDErcited by 112
  11. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
  12. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 90
  13. 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
  14. 2022Music Source Separation with Band-split RNNcited by 69
  15. 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
  16. 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
  17. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  18. 2021Can Audio Captions Be Evaluated with Image Caption Metrics?cited by 33
  19. 2021Sequence Level Contrastive Learning for Text Summarizationcited by 31
  20. 2023Prefix tuning for automated audio captioningcited by 26
  21. 2023StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesiscited by 21
  22. 2019Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoderscited by 18
  23. 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
  24. 2024Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Promptcited by 12
  25. 2023StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Modelscited by 7
  26. 2021Melon Playlist Dataset: a public dataset for audio-based playlist generation and music taggingcited by 5
  27. 2023PromptSpeaker: Speaker Generation Based on Text Descriptionscited by 5
  28. 2019Approximating Human Judgment of Generated Image Qualitycited by 4

Built on by

Not cited yet.