AlignBeat: A Latent Variable Model for Multi-Class Beat Tracking from Partially Labeled Data

Recent neural beat trackers predict beats and downbeats with two independent frame-wise binary classification heads, a convention adopted because a single multi-class head cannot fully exploit datasets that annotate only beats. The heads can disagree, so Beat This moves each predicted downbeat to the nearest predicted beat. We instead predict a sparse set of grid points, each with an event time and one distribution over downbeat, beat and no event, so a downbeat is a beat by construction and no reconciliation is needed. The alignment between grid points and annotated events is latent, and we fit the model by expectation-maximization. Where no downbeat labels were recorded the event class is latent too and is marginalized out. Over eight-fold cross-validation on eighteen datasets, retaining rather than discarding the beat-only labeled data raises beat CMLt by 4.6 points on the datasets that provide it; downbeat CMLt rises 3.6 points over Beat This, with both F-measures moving by less than one point and no post-processing at any stage.