# Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations

arXiv: https://arxiv.org/abs/2412.18940 · HTML: https://mlsanity.com/paper/2412.18940/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Dec 2024.

## Built on by (most cited first, 9 of 9)

- [Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting](https://mlsanity.com/paper/2510.04401/) (2025)
- [MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Models](https://mlsanity.com/paper/2511.03942/) (2025)
- [Stemphonic: All-at-once Flexible Multi-stem Music Generation](https://mlsanity.com/paper/2602.09891/) (2026)
- [CoLyricist: Enhancing Lyric Writing with AI through Workflow-Aligned Support](https://mlsanity.com/paper/2602.22606/) (2026)
- [Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators](https://mlsanity.com/paper/2605.22717/) (2026)
- [What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Music](https://mlsanity.com/paper/2607.00641/) (2026)
- [PaperTok: Exploring the Use of Generative AI for Creating Short-form Videos for Research Communication](https://mlsanity.com/paper/2601.18218/) (2026)
- [WordCraft: Scaffolding the Keyword Method for L2 Vocabulary Learning with Multimodal LLMs](https://mlsanity.com/paper/2602.00762/) (2026)
- [Decomposer: Learning to Decompile Symbolic Music to Programs](https://mlsanity.com/paper/2607.01849/) (2026)

## Abstract (arXiv)

Songwriting is often driven by multimodal inspirations, such as imagery, narratives, or existing music, yet songwriters remain unsupported by current music AI systems in incorporating these multimodal inputs into their creative processes. We introduce Amuse, a songwriting assistant that transforms multimodal (image, text, or audio) inputs into chord progressions that can be seamlessly incorporated into songwriters' creative processes. A key feature of Amuse is its novel method for generating coherent chords that are relevant to music keywords in the absence of datasets with paired examples of multimodal inputs and chords. Specifically, we propose a method that leverages multimodal large language models (LLMs) to convert multimodal inputs into noisy chord suggestions and uses a unimodal chord model to filter the suggestions. A user study with songwriters shows that Amuse effectively supports transforming multimodal ideas into coherent musical suggestions, enhancing users' agency and creativity throughout the songwriting process.