# In a Silent Way: Communication Between AI and Improvising Musicians Beyond Sound

arXiv: https://arxiv.org/abs/1902.06442 · HTML: https://mlsanity.com/paper/1902.06442/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2019.

## Built on by (most cited first, 11 of 11)

- [Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems](https://mlsanity.com/paper/2204.07666/) (2022)
- [Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making](https://mlsanity.com/paper/2101.05303/) (2021)
- [Explainable Computational Creativity](https://mlsanity.com/paper/2205.05682/) (2022)
- [Exploring XAI for the Arts: Explaining Latent Space in Generative Music](https://mlsanity.com/paper/2308.05496/) (2023)
- [Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI](https://mlsanity.com/paper/2311.08336/) (2023)
- [Crossing the Tepper Line: An Emerging Ontology for Describing the Dynamic Sociality of Embodied AI](https://mlsanity.com/paper/2103.08079/) (2021)
- [Improving User Experience with FAICO: Towards a Framework for AI Communication in Human-AI Co-Creativity](https://mlsanity.com/paper/2504.02526/) (2025)
- [Towards the Generation of Musical Explanations with GPT-3](https://mlsanity.com/paper/2206.08264/) (2022)
- [A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Health](https://mlsanity.com/paper/2411.02594/) (2024)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [LK Jam: System Architecture and Implementation of a Real-Time Human-AI Interactive Music Generation System using Role-Aware GRU](https://mlsanity.com/paper/2606.21018/) (2026)

## Abstract (arXiv)

Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn't exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.