LK Jam: System Architecture and Implementation of a Real-Time Human-AI Interactive Music Generation System using Role-Aware GRU
Not on the map. Why?
Abstract (arXiv)
As artificial intelligence advances into the era of Embodied AI, live musical interaction urgently needs to break free from the limitations of offline, unidirectional generation, achieving a "virtual synergy" capable of low-latency, dynamic interplay. To address this, this technical report presents LKJam, a real-time, bidirectional human-computer interactive music generation system based on a lightweight Gated Recurrent Unit (GRU) and a high-performance audio host architecture. In the algorithmic representation layer, this system abandons the computationally expensive fixed time-grid. Instead, it constructs a multi-dimensional sparse event stream integrating time-shifts, continuous harmonic embeddings, and role-aware encoding, enabling the model to accurately capture turn-taking logic and micro-timing in a single-step inference. In the engineering implementation layer, this paper builds a strict multithreaded lock-free communication bridge using C++ and the JUCE framework, incorporating the RTNeural inference engine designed specifically for real-time audio. By utilizing compile-time network topology solidification and a zero-allocation (allocation-free) mechanism, the end-to-end overhead of autoregressive decoding is strictly locked at (O(1)) complexity, structurally mitigating the risk of audio thread dropouts in DAW plugin environments. Furthermore, this study designs a three-stage progressive training strategy, achieving a leap from basic chord harmonization to expert-level interaction. Preliminary observations and architectural analysis demonstrate that while ensuring musical coherence and interactive role-play, the proposed system successfully challenges extreme real-time engineering constraints, offering a highly robust and deployable technical paradigm for next-generation AI co-performers in live music.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 19 references that are in the map. First posted on arXiv 19 Jun 2026.
Builds on
10 of 19 references.
References · 10
- 2017Attention Is All You Needcited by 60,361
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
- 2018This Time with Feeling: Learning Expressive Musical Performancecited by 68
- 2019In a Silent Way: Communication Between AI and Improvising Musicians Beyond Soundcited by 11
- 2024A Survey of Music Generation in the Context of Interactioncited by 2
- 2025Incorporating Structure and Chord Constraints in Symbolic Transformer-based Melodic Harmonizationcited by 2
- 2024Practical and Reproducible Symbolic Music Generation by Large Language Models with Structural Embeddingscited by 1
Built on by
Not cited yet.