mlsanityData to 7 Oct 2026

LP-MusicCaps: LLM-Based Pseudo Music Captioning

arXiv 2307.16372 ↗First posted 31 Jul 2023cs.SD cs.IR cs.MMcited by 83

Not on the map. Why?

Abstract (arXiv)

Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its importance, researchers face challenges due to the costly and time-consuming collection process of existing music-language datasets, which are limited in size. To address this data scarcity issue, we propose the use of large language models (LLMs) to artificially generate the description sentences from large-scale tag datasets. This results in approximately 2.2M captions paired with 0.5M audio clips. We term it Large Language Model based Pseudo music caption dataset, shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale music captioning dataset with various quantitative evaluation metrics used in the field of natural language processing as well as human evaluation. In addition, we trained a transformer-based music captioning model with the dataset and evaluated it under zero-shot and transfer-learning settings. The results demonstrate that our proposed approach outperforms the supervised baseline model.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jul 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 35; 2025: 17; 2026: 22.

83 papers.

Sort:

By area:

  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2025Qwen2.5-Omni Technical Reportcited by 704
  3. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  4. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  5. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  6. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  7. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  8. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
  9. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
  10. 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
  11. 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
  12. 2023MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Responsecited by 45
38 more
  1. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
  2. 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 40
  3. 2024Foundation Models for Music: A Surveycited by 34
  4. 2023LLark: A Multimodal Instruction-Following Language Model for Musiccited by 29
  5. 2025SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generationcited by 29
  6. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  7. 2024MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussionscited by 23
  8. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  9. 2026Large Language Model Reasoning Failurescited by 20
  10. 2024MidiCaps: A large-scale MIDI dataset with text captionscited by 19
  11. 2024UniMuMo: Unified Text, Music and Motion Generationcited by 19
  12. 2024FLUX that Plays Musiccited by 18
  13. 2023Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editingcited by 15
  14. 2025CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Followingcited by 15
  15. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 15
  16. 2024Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompanimentcited by 13
  17. 2024MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptionscited by 13
  18. 2024CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Modelscited by 12
  19. 2024Accompanied Singing Voice Synthesis with Fully Text-controlled Melodycited by 9
  20. 2024OpenMU: Your Swiss Army Knife for Music Understandingcited by 9
  21. 2024Audio Dialogues: Dialogues dataset for audio and music understandingcited by 7
  22. 2026SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processingcited by 7
  23. 2024Quality-aware Masked Diffusion Transformer for Enhanced Music Generationcited by 6
  24. 2025MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Taskscited by 6
  25. 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
  26. 2024Futga: Towards Fine-grained Music Understanding through Temporally-enhanced Generative Augmentationcited by 5
  27. 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 5
  28. 2024LightHouse: A Survey of AGI Hallucinationcited by 4
  29. 2024I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognitioncited by 4
  30. 2024CoLLAP: Contrastive Long-form Language-Audio Pretraining with Musical Temporal Structure Augmentationcited by 4
  31. 2025A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectivescited by 4
  32. 2025Versatile Framework for Song Generation with Prompt-based Controlcited by 4
  33. 2023WikiMuTe: A web-sourced dataset of semantic descriptions for music audiocited by 3
  34. 2024Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learningcited by 3
  35. 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Modelscited by 2
  36. 2025DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuningcited by 2
  37. 2025Exploring State-Space-Model based Language Model in Music Generationcited by 2
  38. 2025Language-Guided Contrastive Audio-Visual Masked Autoencoder with Automatically Generated Audio-Visual-Text Triplets from Videoscited by 2

50 most cited of 83 All 83 (JSON)