mlsanityData to 7 Oct 2026

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

arXiv 1803.05428 ↗First posted 13 Mar 2018cs.LG cs.SD eess.AScited by 131

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The Variational Autoencoder (VAE) has proven to be an effective model for producing semantically meaningful latent representations for natural data. However, it has thus far seen limited application to sequential data, and, as we demonstrate, existing recurrent VAE models have difficulty modeling sequences with long-term structure. To address this issue, we propose the use of a hierarchical decoder, which first outputs embeddings for subsequences of the input and then uses these embeddings to generate each subsequence independently. This structure encourages the model to utilize its latent code, thereby avoiding the "posterior collapse" problem, which remains an issue for recurrent VAEs. We apply this architecture to modeling sequences of musical notes and find that it exhibits dramatically better sampling, interpolation, and reconstruction performance than a "flat" baseline model. An implementation of our "MusicVAE" is available online at http://g.co/magenta/musicvae-code.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 21; 2023: 27; 2024: 24; 2025: 20; 2026: 13.

131 papers.

Sort:

By area:

  1. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  2. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  3. 2024Symbolic Music Generation with Non-Differentiable Rule Guided Diffusioncited by 52
  4. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  5. 2022Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generationcited by 48
  6. 2024SNAC: Multi-Scale Neural Audio Codeccited by 42
  7. 2021MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAEcited by 39
  8. 2022Symphony Generation with Permutation Invariant Language Modelcited by 35
  9. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
  10. 2024Foundation Models for Music: A Surveycited by 34
  11. 2021Controllable deep melody generation via hierarchical music structure representationcited by 30
  12. 2022Multitrack Music Transformercited by 28
38 more
  1. 2021Creativity and Machine Learning: A Surveycited by 21
  2. 2021Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Featurescited by 19
  3. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  4. 2023V2Meow: Meowing to the Visual Beat via Video-to-Music Generationcited by 18
  5. 2022A Domain-Knowledge-Inspired Music Embedding Space and a Novel Attention Mechanism for Symbolic Music Modelingcited by 17
  6. 2023Long-Term Rhythmic Video Soundtrackercited by 17
  7. 2023Discrete Diffusion Probabilistic Models for Symbolic Music Generationcited by 16
  8. 2021Neural Distributed Source Codingcited by 13
  9. 2021Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structurecited by 12
  10. 2021Music Composition with Deep Learning: A Reviewcited by 12
  11. 2021Learning Interpretable Representations of Entanglement in Quantum Optics Experiments using Deep Generative Modelscited by 12
  12. 2022Symbolic music generation conditioned on continuous-valued emotionscited by 12
  13. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  14. 2021Structure-Enhanced Pop Music Generation via Harmony-Aware Learningcited by 11
  15. 2022A Review of Intelligent Music Generation Systemscited by 11
  16. 2024DExter: Learning and Controlling Performance Expression with Diffusion Modelscited by 11
  17. 2024Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Surveycited by 10
  18. 2025Survey on the Evaluation of Generative Models in Musiccited by 10
  19. 2023Composer's Assistant: An Interactive Transformer for Multi-Track MIDI Infillingcited by 9
  20. 2023A Survey of AI Music Generation Tools and Modelscited by 9
  21. 2023Exploring XAI for the Arts: Explaining Latent Space in Generative Musiccited by 8
  22. 2023Composer Style-specific Symbolic Music Generation Using Vector Quantized Discrete Diffusion Modelscited by 8
  23. 2024Navigating Tabular Data Synthesis Research: Understanding User Needs and Tool Capabilitiescited by 8
  24. 2021Is Disentanglement enough? On Latent Representations for Controllable Music Generationcited by 7
  25. 2022Variational Laplace Autoencoderscited by 7
  26. 2023Generating symbolic music using diffusion modelscited by 7
  27. 2024Creativity in AI: Progresses and Challengescited by 7
  28. 2025Scaling Self-Supervised Representation Learning for Symbolic Piano Performancecited by 7
  29. 2021Level Generation for Angry Birds with Sequential VAE and Latent Variable Evolutioncited by 6
  30. 2022Autoencoding Low-Resolution MRI for Semantically Smooth Interpolation of Anisotropic MRIcited by 6
  31. 2023A Comprehensive Survey for Evaluation Methodologies of AI-Generated Musiccited by 6
  32. 2024An Order-Complexity Aesthetic Assessment Model for Aesthetic-aware Music Recommendationcited by 6
  33. 2024Motifs, Phrases, and Beyond: The Modelling of Structure in Symbolic Music Generationcited by 6
  34. 2024Bridging Paintings and Music -- Exploring Emotion based Music Generation through Paintingscited by 6
  35. 2022Learning long-term music representations via hierarchical contextual constraintscited by 5
  36. 2022Subjective Evaluation of Deep Learning Models for Symbolic Music Compositioncited by 5
  37. 2022The Power of Fragmentation: A Hierarchical Transformer Model for Structural Segmentation in Symbolic Music Generationcited by 5
  38. 2024Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Controlcited by 5

50 most cited of 131 All 131 (JSON)