# RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2002.03082 · HTML: https://mlsanity.com/paper/2002.03082/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2020.

## Built on by (most cited first, 19 of 19)

- [MusicRL: Aligning Music Generation to Human Preferences](https://mlsanity.com/paper/2402.04229/) (2024)
- [NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms](https://mlsanity.com/paper/2502.18008/) (2025)
- [Creativity and Machine Learning: A Survey](https://mlsanity.com/paper/2104.02726/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [SongDriver: Real-time Music Accompaniment Generation without Logical Latency nor Exposure Bias](https://mlsanity.com/paper/2209.06054/) (2022)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges](https://mlsanity.com/paper/2308.00031/) (2023)
- [Live Music Models](https://mlsanity.com/paper/2508.04651/) (2025)
- [ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers](https://mlsanity.com/paper/2502.21267/) (2025)
- [REMAST: Real-time Emotion-based Music Arrangement with Soft Transition](https://mlsanity.com/paper/2305.08029/) (2023)
- [CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN](https://mlsanity.com/paper/2104.00353/) (2021)
- [Adaptive Accompaniment with ReaLchords](https://mlsanity.com/paper/2506.14723/) (2025)
- [A Survey on Artificial Intelligence for Music Generation: Agents, Domains and Perspectives](https://mlsanity.com/paper/2210.13944/) (2022)
- [Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction](https://mlsanity.com/paper/2511.17879/) (2025)
- [In-depth analysis of music structure as a text network](https://mlsanity.com/paper/2303.13631/) (2023)
- [Evaluating Co-Creativity using Total Information Flow](https://mlsanity.com/paper/2402.06810/) (2024)
- [Reward Dropout Improves Control: Bi-objective Perspective on Reinforced LM](https://mlsanity.com/paper/2310.04483/) (2023)
- [A novel LSTM music generator based on the fractional time-frequency feature extraction](https://mlsanity.com/paper/2604.17823/) (2026)
- [JazzSAMBA: A Synchronous and Asynchronous Multi-take Band Audio Dataset of Jazz Standards for Live Music Models](https://mlsanity.com/paper/2609.34931/) (2026)

## Abstract (arXiv)

This paper presents a deep reinforcement learning algorithm for online accompaniment generation, with potential for real-time interactive human-machine duet improvisation. Different from offline music generation and harmonization, online music accompaniment requires the algorithm to respond to human input and generate the machine counterpart in a sequential order. We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state). The key of this algorithm is the well-functioning reward model. Instead of defining it using music composition rules, we learn this model from monophonic and polyphonic training data. This model considers the compatibility of the machine-generated note with both the machine-generated context and the human-generated context. Experiments show that this algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part. Subjective evaluations on preferences show that the proposed algorithm generates music pieces of higher quality than the baseline method.