# Just Ask for Music (JAM): Multimodal and Personalized Natural Language Music Recommendation

arXiv: https://arxiv.org/abs/2507.15826 · HTML: https://mlsanity.com/paper/2507.15826/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [A Survey on Conversational Recommender Systems](https://mlsanity.com/paper/2004.00646/) (2020)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Translation-based Recommendation](https://mlsanity.com/paper/1707.02410/) (2017)
- [Personalized Audiobook Recommendations at Spotify Through Graph Neural Networks](https://mlsanity.com/paper/2403.05185/) (2024)
- [A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios](https://mlsanity.com/paper/2409.17864/) (2024)
- [Beyond Single Items: Exploring User Preferences in Item Sets with the Conversational Playlist Curation Dataset](https://mlsanity.com/paper/2303.06791/) (2023)
- [Hierarchical Latent Relation Modeling for Collaborative Metric Learning](https://mlsanity.com/paper/2108.04655/) (2021)
- [Melon Playlist Dataset: a public dataset for audio-based playlist generation and music tagging](https://mlsanity.com/paper/2102.00201/) (2021)
- [An Experimental Comparison Of Multi-view Self-supervised Methods For Music Tagging](https://mlsanity.com/paper/2404.09177/) (2024)
- [Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session Recommendation](https://mlsanity.com/paper/2408.16578/) (2024)
- [Music Discovery Dialogue Generation Using Human Intent Analysis and Large Language Models](https://mlsanity.com/paper/2411.07439/) (2024)
- [Modeling Activity-Driven Music Listening with PACE](https://mlsanity.com/paper/2405.01417/) (2024)
- [Harnessing High-Level Song Descriptors towards Natural Language-Based Music Recommendation](https://mlsanity.com/paper/2411.05649/) (2024)

## Built on by (most cited first, 5 of 5)

- [TalkPlayData 2: An Agentic Synthetic Data Pipeline for Multimodal Conversational Music Recommendation](https://mlsanity.com/paper/2509.09685/) (2025)
- [WeMusic-Agent: Efficient Conversational Music Recommendation via Knowledge Internalization and Agentic Boundary Learning](https://mlsanity.com/paper/2512.16108/) (2025)
- [Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluation](https://mlsanity.com/paper/2511.16478/) (2025)
- [Beyond Musical Descriptors: Extracting Preference-Bearing Intent in Music Queries](https://mlsanity.com/paper/2602.12301/) (2026)
- [Two Views, One Voice: Evidence-Grounded Conversational Music Recommendation](https://mlsanity.com/paper/2607.24846/) (2026)

## Abstract (arXiv)

Natural language interfaces offer a compelling approach for music recommendation, enabling users to express complex preferences conversationally. While Large Language Models (LLMs) show promise in this direction, their scalability in recommender systems is limited by high costs and latency. Retrieval-based approaches using smaller language models mitigate these issues but often rely on single-modal item representations, overlook long-term user preferences, and require full model retraining, posing challenges for real-world deployment. In this paper, we present JAM (Just Ask for Music), a lightweight and intuitive framework for natural language music recommendation. JAM models user-query-item interactions as vector translations in a shared latent space, inspired by knowledge graph embedding methods like TransE. To capture the complexity of music and user intent, JAM aggregates multimodal item features via cross-attention and sparse mixture-of-experts. We also introduce JAMSessions, a new dataset of over 100k user-query-item triples with anonymized user/item embeddings, uniquely combining conversational queries and user long-term preferences. Our results show that JAM provides accurate recommendations, produces intuitive representations suitable for practical use cases, and can be easily integrated with existing music recommendation stacks.