# Masked diffusion enables coherent beat tracking

arXiv: https://arxiv.org/abs/2608.04624 · HTML: https://mlsanity.com/paper/2608.04624/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 18 of 18)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://mlsanity.com/paper/2409.02908/) (2024)
- [Beat this! Accurate beat tracking without DBN postprocessing](https://mlsanity.com/paper/2407.21658/) (2024)
- [SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond](https://mlsanity.com/paper/2406.17672/) (2024)
- [BeatFM: Improving Beat Tracking with Pre-trained Music Foundation Model](https://mlsanity.com/paper/2508.09790/) (2025)
- [Controlling Contrastive Self-Supervised Learning with Knowledge-Driven Multiple Hypothesis: Application to Beat Tracking](https://mlsanity.com/paper/2510.25560/) (2025)
- [An Analysis Method for Metric-Level Switching in Beat Tracking](https://mlsanity.com/paper/2210.06817/) (2022)
- [Skip That Beat: Augmenting Meter Tracking Models for Underrepresented Time Signatures](https://mlsanity.com/paper/2502.12972/) (2025)
- [Beat and Downbeat Tracking in Performance MIDI Using an End-to-End Transformer Architecture](https://mlsanity.com/paper/2507.00466/) (2025)
- [Music Boomerang: Reusing Diffusion Models for Data Augmentation and Audio Manipulation](https://mlsanity.com/paper/2507.04864/) (2025)
- [HingeNet: A Harmonic-Aware Fine-Tuning Approach for Beat Tracking](https://mlsanity.com/paper/2508.09788/) (2025)

## Abstract (arXiv)

Current neural networks for beat tracking generate invalid outputs, such as consecutive downbeats and erratic tempo changes, even when these are not present in the training data. Heavy post-processing techniques can alleviate these problems, but the original cause of this inconsistent behaviour remains unknown. We hypothesise that it stems from inadequate modelling of multiple plausible output beat grids, resulting in an invalid mixture of competing interpretations. We propose a masked diffusion approach that properly models multiple outputs and enables the model to build coherent predictions through iterative inference. We devise three modifications to standard masked diffusion that enable its application to beat tracking: independent masking of beats and downbeats during training and inference, a balanced masking scheduler for inference, and peak-picking across inference steps. Our approach reduces erratic behaviours and improves beat-tracking performance.