S2Cap: A Benchmark and a Baseline for Singing Style Captioning
Not on the map. Why?
Abstract (arXiv)
Singing voices contain much richer information than common voices, including varied vocal and acoustic properties. However, current open-source audio-text datasets for singing voices capture only a narrow range of attributes and lack acoustic features, leading to limited utility towards downstream tasks, such as style captioning. To fill this gap, we formally define the singing style captioning task and present S2Cap, a dataset of singing voices with detailed descriptions covering diverse vocal, acoustic, and demographic characteristics. Using this dataset, we develop an efficient and straightforward baseline algorithm for singing style captioning. The dataset is available at https://zenodo.org/records/15673764.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 15 Sep 2024.
Builds on
40 of 68 references.
References · 40
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
28 more
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
- 2021AST: Audio Spectrogram Transformercited by 634
- 2019Clotho: An Audio Captioning Datasetcited by 354
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2023Pengi: An Audio Language Model for Audio Taskscited by 217
- 2016Movie Descriptioncited by 180
- 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
- 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
- 2022Hybrid Transformers for Music Source Separationcited by 134
- 2016Improved Image Captioning via Policy Gradient optimization of SPIDErcited by 112
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
- 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 90
- 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
- 2022Music Source Separation with Band-split RNNcited by 69
- 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
- 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
- 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
- 2021Can Audio Captions Be Evaluated with Image Caption Metrics?cited by 33
- 2021Sequence Level Contrastive Learning for Text Summarizationcited by 31
- 2023Prefix tuning for automated audio captioningcited by 26
- 2023StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesiscited by 21
- 2019Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoderscited by 18
- 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
- 2024Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Promptcited by 12
- 2023StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Modelscited by 7
- 2021Melon Playlist Dataset: a public dataset for audio-based playlist generation and music taggingcited by 5
- 2023PromptSpeaker: Speaker Generation Based on Text Descriptionscited by 5
- 2019Approximating Human Judgment of Generated Image Qualitycited by 4
Built on by
Not cited yet.