mlsanityData to 7 Oct 2026

MuseChat: A Conversational Music Recommendation System for Videos

arXiv 2310.06282 ↗First posted 10 Oct 2023cs.LG cs.CV cs.IRcited by 21

Not on the map. Why?

Abstract (arXiv)

Music recommendation for videos attracts growing interest in multi-modal research. However, existing systems focus primarily on content compatibility, often ignoring the users' preferences. Their inability to interact with users for further refinements or to provide explanations leads to a less satisfying experience. We address these issues with MuseChat, a first-of-its-kind dialogue-based recommendation system that personalizes music suggestions for videos. Our system consists of two key functionalities with associated modules: recommendation and reasoning. The recommendation module takes a video along with optional information including previous suggested music and user's preference as inputs and retrieves an appropriate music matching the context. The reasoning module, equipped with the power of Large Language Model (Vicuna-7B) and extended to multi-modal inputs, is able to provide reasonable explanation for the recommended music. To evaluate the effectiveness of MuseChat, we build a large-scale dataset, conversational music recommendation for videos, that simulates a two-turn interaction between a user and a recommender based on accurate music track information. Experiment results show that MuseChat achieves significant improvements over existing video-based music retrieval methods as well as offers strong interpretability and interactability.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 57 of its 76 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

57 of 76 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  10. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  11. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  12. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  4. 2019Contrastive Multiview Codingcited by 1,297
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  7. 2021AST: Audio Spectrogram Transformercited by 634
  8. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  9. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  10. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  11. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  12. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  13. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  14. 2020A Survey on Conversational Recommender Systemscited by 142
  15. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  16. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  17. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 114
  18. 2021Talk-to-Edit: Fine-Grained Facial Editing via Dialogcited by 102
  19. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 85
  20. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
  21. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  22. 2019musicnn: Pre-trained convolutional neural networks for music audio taggingcited by 49
  23. 2023Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Modelscited by 48
  24. 2016Automatic tagging using deep convolutional neural networkscited by 39
  25. 2020Evaluation of CNN-based Automatic Music Tagging Modelscited by 39
  26. 2023ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMscited by 35
  27. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  28. 2022A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systemscited by 32
  29. 2022Contrastive Audio-Language Learning for Musiccited by 32
  30. 2017Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveformscited by 27
  31. 2021Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Informationcited by 26
  32. 2021Semi-Supervised Music Tagging Transformercited by 24
  33. 2022It's Time for Artistic Correspondence in Music and Videocited by 18
  34. 2022Toward Universal Text-to-Music Retrievalcited by 18
  35. 2023SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Modelcited by 18
  36. 2023Language-Guided Music Recommendation for Video via Prompt Analogiescited by 18
  37. 2019Audio-Visual Embedding for Cross-Modal MusicVideo Retrieval through Supervised Deep CCAcited by 16
  38. 2020Disentangled Multidimensional Metric Learning for Music Similaritycited by 16

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 6; 2026: 3.

21 papers.

Sort:

By area:

  1. 2024Tell What You Hear From What You See -- Video to Audio Generation Through Textcited by 34
  2. 2024From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generationcited by 23
  3. 2024UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Modelscited by 18
  4. 2024ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extractioncited by 17
  5. 2025HarmonySet: A Comprehensive Dataset for Understanding Video-Music Semantic Alignment and Temporal Synchronizationcited by 14
  6. 2024EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLMcited by 12
  7. 2024Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizerscited by 7
  8. 2024Hedge Fund Portfolio Construction Using PolyModel Theory and iTransformercited by 7
  9. 2025AutoMV: An Automatic Multi-Agent System for Music Video Generationcited by 7
  10. 2024Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairnesscited by 6
  11. 2024Futga: Towards Fine-grained Music Understanding through Temporally-enhanced Generative Augmentationcited by 5
  12. 2024Mamba Fusion: Learning Actions Through Questioningcited by 5
9 more
  1. 2025FilmComposer: LLM-Driven Music Production for Silent Film Clipscited by 5
  2. 2024Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidancecited by 4
  3. 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understandingcited by 3
  4. 2026CPGRec+: A Balance-oriented Framework for Personalized Video Game Recommendationscited by 3
  5. 2024Character-based Outfit Generation with Vision-augmented Style Extraction via LLMscited by 1
  6. 2025ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Predictioncited by 1
  7. 2025ChromouVQA: Benchmarking Vision-Language Models under Chromatic Camouflaged Imagescited by 1
  8. 2026DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialoguescited by 0
  9. 2026Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysiscited by 0