# Estimating Musical Surprisal from Audio in Autoregressive Diffusion Model Noise Spaces

arXiv: https://arxiv.org/abs/2508.05306 · HTML: https://mlsanity.com/paper/2508.05306/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [GIVT: Generative Infinite-Vocabulary Transformers](https://mlsanity.com/paper/2312.02116/) (2023)
- [Music transcription modelling and composition using deep learning](https://mlsanity.com/paper/1604.08723/) (2016)
- [Music2Latent: Consistency Autoencoders for Latent Audio Compression](https://mlsanity.com/paper/2408.06500/) (2024)
- [Ambient Diffusion Omni: Training Good Models with Bad Data](https://mlsanity.com/paper/2506.10038/) (2025)
- [Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation](https://mlsanity.com/paper/2411.18447/) (2024)
- [Controlling Surprisal in Music Generation via Information Content Curve Matching](https://mlsanity.com/paper/2408.06022/) (2024)
- [Exploring Sampling Techniques for Generating Melodies with a Transformer Language Model](https://mlsanity.com/paper/2308.09454/) (2023)
- [Deep Generative Models of Music Expectation](https://mlsanity.com/paper/2310.03500/) (2023)
- [Estimating Musical Surprisal in Audio](https://mlsanity.com/paper/2501.07474/) (2025)
- [Deep Music Information Dynamics](https://mlsanity.com/paper/2102.01133/) (2021)

## Built on by (most cited first, 1 of 1)

- [Perceptually Aligning Representations of Music via Noise-Augmented Autoencoders](https://mlsanity.com/paper/2511.05350/) (2025)

## Abstract (arXiv)

Recently, the information content (IC) of predictions from a Generative Infinite-Vocabulary Transformer (GIVT) has been used to model musical expectancy and surprisal in audio. We investigate the effectiveness of such modelling using IC calculated with autoregressive diffusion models (ADMs). We empirically show that IC estimates of models based on two different diffusion ordinary differential equations (ODEs) describe diverse data better, in terms of negative log-likelihood, than a GIVT. We evaluate diffusion model IC's effectiveness in capturing surprisal aspects by examining two tasks: (1) capturing monophonic pitch surprisal, and (2) detecting segment boundaries in multi-track audio. In both tasks, the diffusion models match or exceed the performance of a GIVT. We hypothesize that the surprisal estimated at different diffusion process noise levels corresponds to the surprisal of music and audio features present at different audio granularities. Testing our hypothesis, we find that, for appropriate noise levels, the studied musical surprisal tasks' results improve. Code is provided on github.com/SonyCSLParis/audioic.