# Affective Music Recommendation: A Rollout-Based World Model for Offline Preference Optimization

arXiv: https://arxiv.org/abs/2605.28810 · HTML: https://mlsanity.com/paper/2605.28810/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 13 of 13)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [RecSim: A Configurable Simulation Platform for Recommender Systems](https://mlsanity.com/paper/1909.04847/) (2019)
- [Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation](https://mlsanity.com/paper/2307.04571/) (2023)
- [KuaiSim: A Comprehensive Simulator for Recommender Systems](https://mlsanity.com/paper/2309.12645/) (2023)
- [CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages](https://mlsanity.com/paper/2502.10362/) (2025)
- [Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2110.15097/) (2021)
- [Towards Unified Music Emotion Recognition across Dimensional and Categorical Models](https://mlsanity.com/paper/2502.03979/) (2025)
- [A Survey of Affective Recommender Systems: Modeling Attitudes, Emotions, and Moods for Personalization](https://mlsanity.com/paper/2508.20289/) (2025)
- [Causally-Informed Reinforcement Learning for Adaptive Emotion-Aware Social Media Recommendation](https://mlsanity.com/paper/2511.14768/) (2025)

## Abstract (arXiv)

Functional music applications, from consumer focus and sleep aids to clinical interventions, share a distinctive recommendation problem: success is defined by the listener's affective state, but online experimentation on emotion is ethically constrained, particularly for clinical populations who cannot reliably skip a song or report distress. We describe AMRS, the Affective Music Recommendation System deployed on LUCID's health-and-wellness platforms, which serve clinical users (primarily older adults with neurocognitive conditions) and consumer-wellness users across energize, focus, calm, and sleep modes. AMRS is built around a rollout-based world model: a causal transformer trained on logged listening data to jointly predict engagement, binary rating, and self-reported valence and arousal. The world model serves both as an in-silico simulator for offline policy training and as a stress-testing tool before deployment. A recommender policy initialized by behaviour cloning is fine-tuned offline with Direct Preference Optimization (DPO) against a configurable multi-objective utility function. Under a strict cold-start protocol, the world model predicts both behavioural and affective signals with usable fidelity; DPO improves predicted valence and arousal over the cloned baseline while maintaining a similar diversity profile and avoiding the distributional collapse produced by greedy optimization. We position the work as an early deployed validation of a methodology for affective recommendation when online experimentation is ethically untenable.