mlsanityData to 7 Oct 2026

Sing-On-Your-Beat: Simple Text-Controllable Accompaniment Generations

arXiv 2411.01661 ↗First posted 3 Nov 2024cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Singing is one of the most cherished forms of human entertainment. However, creating a beautiful song requires an accompaniment that complements the vocals and aligns well with the song instruments and genre. With advancements in deep learning, previous research has focused on generating suitable accompaniments but often lacks precise alignment with the desired instrumentation and genre. To address this, we propose a straightforward method that enables control over the accompaniment through text prompts, allowing the generation of music that complements the vocals and aligns with the song instrumental and genre requirements. Through extensive experiments, we successfully generate 10-second accompaniments using vocal input and text control.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 3 Nov 2024.

Builds on

17 of 24 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  6. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  7. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  8. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  9. 2020Jukebox: A Generative Model for Musiccited by 535
  10. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
  11. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  12. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
5 more
  1. 2022Hybrid Transformers for Music Source Separationcited by 134
  2. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  3. 2021Hybrid Spectrogram and Waveform Source Separationcited by 90
  4. 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
  5. 2024FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generationcited by 4

Built on by

Not cited yet.