# Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing

arXiv: https://arxiv.org/abs/2101.08846 · HTML: https://mlsanity.com/paper/2101.08846/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2021.

## Built on by (most cited first, 6 of 6)

- [Generative Disco: Text-to-Video Generation for Music Visualization](https://mlsanity.com/paper/2304.08551/) (2023)
- [TutoAI: A Cross-domain Framework for AI-assisted Mixed-media Tutorial Creation on Physical Tasks](https://mlsanity.com/paper/2403.08049/) (2024)
- [ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference](https://mlsanity.com/paper/2509.14537/) (2025)
- [Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs](https://mlsanity.com/paper/2405.12946/) (2024)
- [NoteIt: A System Converting Instructional Videos to Interactable Notes Through Multimodal Video Understanding](https://mlsanity.com/paper/2508.14395/) (2025)
- [Adaptive Segmentation of EEG for Machine Learning Applications](https://mlsanity.com/paper/2508.20336/) (2025)

## Abstract (arXiv)

Learning musical instruments using online instructional videos has become increasingly prevalent. However, pre-recorded videos lack the instantaneous feedback and personal tailoring that human tutors provide. In addition, existing video navigations are not optimized for instrument learning, making the learning experience encumbered. Guided by our formative interviews with guitar players and prior literature, we designed Soloist, a mixed-initiative learning framework that automatically generates customizable curriculums from off-the-shelf guitar video lessons. Soloist takes raw videos as input and leverages deep-learning based audio processing to extract musical information. This back-end processing is used to provide an interactive visualization to support effective video navigation and real-time feedback on the user's performance, creating a guided learning experience. We demonstrate the capabilities and specific use-cases of Soloist within the domain of learning electric guitar solos using instructional YouTube videos. A remote user study, conducted to gather feedback from guitar players, shows encouraging results as the users unanimously preferred learning with Soloist over unconverted instructional videos.