mlsanityData to 14 Oct 2026

TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization

arXiv 2509.00914 ↗First posted 31 Aug 2025cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

The success of the generative model has gained unprecedented attention in the music generation area. Transformer-based architectures have set new benchmarks for model performance. However, their practical adoption is hindered by some critical challenges: the demand for massive computational resources and inference time, due to their large number of parameters. These obstacles make them infeasible to deploy on edge devices, such as smartphones and wearables, with limited computational resources. In this work, we present TinyMusician, a lightweight music generation model distilled from MusicGen (a State-of-the-art music generation model). TinyMusician integrates two innovations: (i) Stage-mixed Bidirectional and Skewed KL-Divergence and (ii) Adaptive Mixed-Precision Quantization. The experimental results demonstrate that TinyMusician retains 93% of the MusicGen-Small performance with 55% less model size. TinyMusician is the first mobile-deployable music generation model that eliminates cloud dependency while maintaining high audio fidelity and efficient resource usage

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 87 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

57 of 87 references.

References · 57

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2020Knowledge Distillation: A Surveycited by 1,603
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2022High Fidelity Neural Audio Compressioncited by 687
  12. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
38 more
  1. 2019Learned Step Size Quantizationcited by 628
  2. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  3. 2020Jukebox: A Generative Model for Musiccited by 542
  4. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 508
  5. 2022Transformers in Time Series: A Surveycited by 497
  6. 2023Simple and Controllable Music Generationcited by 488
  7. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  8. 2023MusicLM: Generating Music From Textcited by 448
  9. 2022Post-training Quantization on Diffusion Modelscited by 267
  10. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 225
  11. 2019MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentationcited by 195
  12. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  13. 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
  14. 2023Stay on topic with Classifier-Free Guidancecited by 91
  15. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  16. 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 71
  17. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  18. 2023Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memorycited by 64
  19. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generationcited by 64
  20. 2018Online Model Distillation for Efficient Video Inferencecited by 60
  21. 2022Advancing Model Pruning via Bi-level Optimizationcited by 56
  22. 2023Efficient Neural Music Generationcited by 50
  23. 2024A Survey on Transformer Compressioncited by 46
  24. 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusioncited by 46
  25. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  26. 2021One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teacherscited by 40
  27. 2021Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformercited by 40
  28. 2023Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer modelcited by 37
  29. 2025Emergent Abilities in Large Language Models: A Surveycited by 37
  30. 2023Anticipatory Music Transformercited by 36
  31. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  32. 2023Large Language Models for Networking: Applications, Enabling Techniques, and Challengescited by 28
  33. 2022Efficient Large-scale Audio Tagging via Transformer-to-CNN Knowledge Distillationcited by 27
  34. 2024BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillationcited by 19
  35. 2023CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-trainingcited by 18
  36. 2024FLUX that Plays Musiccited by 18
  37. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17
  38. 2023ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Modelscited by 15

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring compressibility of transformer based text-to-music (TTM) models ↗
  2. 2026A Quantized Native Runtime for On-Device Semantic Audio Generation
  3. 2025ACE-Step: A Step Towards Music Generation Foundation Model
  4. 2025Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation
  5. 2026ACE-Step 1.5: Pushing the Boundaries of Open-Source Music Generation
  6. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusion
  7. 2026Training-Efficient Text-to-Music Generation with State-Space Modeling
  8. 2024Long-form music generation with latent diffusion