# All-In-One Metrical And Functional Structure Analysis With Neighborhood Attentions on Demixed Audio

arXiv: https://arxiv.org/abs/2307.16425 · HTML: https://mlsanity.com/paper/2307.16425/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2023.

## Built on by (most cited first, 28 of 28)

- [ACE-Step: A Step Towards Music Generation Foundation Model](https://mlsanity.com/paper/2506.00045/) (2025)
- [Beat this! Accurate beat tracking without DBN postprocessing](https://mlsanity.com/paper/2407.21658/) (2024)
- [LeVo: High-Quality Song Generation with Multi-Preference Alignment](https://mlsanity.com/paper/2506.07520/) (2025)
- [SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement](https://mlsanity.com/paper/2506.07634/) (2025)
- [DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching](https://mlsanity.com/paper/2510.22950/) (2025)
- [Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generation](https://mlsanity.com/paper/2503.19611/) (2025)
- [MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation](https://mlsanity.com/paper/2605.20183/) (2026)
- [SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition](https://mlsanity.com/paper/2402.17645/) (2024)
- [JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability and Aesthetic Alignment](https://mlsanity.com/paper/2507.20880/) (2025)
- [DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing](https://mlsanity.com/paper/2604.04875/) (2026)
- [MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks](https://mlsanity.com/paper/2507.23511/) (2025)
- [SongPrep: A Preprocessing Framework and End-to-end Model for Full-song Structure Parsing and Lyrics Transcription](https://mlsanity.com/paper/2509.17404/) (2025)
- [Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation](https://mlsanity.com/paper/2508.01394/) (2025)
- [PIAST: A Multimodal Piano Dataset with Audio, Symbolic and Text](https://mlsanity.com/paper/2411.02551/) (2024)
- [Music Plagiarism Detection: Problem Formulation and a Segment-based Solution](https://mlsanity.com/paper/2601.21260/) (2026)
- [Low-Resource Guidance for Controllable Latent Audio Diffusion](https://mlsanity.com/paper/2603.04366/) (2026)
- [Real-world Music Plagiarism Detection With Music Segment Transcription System](https://mlsanity.com/paper/2509.08282/) (2025)
- [Do Foundational Audio Encoders Understand Music Structure?](https://mlsanity.com/paper/2512.17209/) (2025)
- [BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation](https://mlsanity.com/paper/2605.27067/) (2026)
- [AI TrackMate: Finally, Someone Who Will Give Your Music More Than Just "Sounds Great!"](https://mlsanity.com/paper/2412.06617/) (2024)
- [Barwise Section Boundary Detection in Symbolic Music Using Convolutional Neural Networks](https://mlsanity.com/paper/2509.16566/) (2025)
- [Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrieval](https://mlsanity.com/paper/2602.15074/) (2026)
- [Unsupervised Evaluation of Deep Audio Embeddings for Music Structure Analysis](https://mlsanity.com/paper/2603.27218/) (2026)
- [SegTune: Structured and Fine-Grained Control for Song Generation](https://mlsanity.com/paper/2606.02638/) (2026)
- [SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling](https://mlsanity.com/paper/2606.03169/) (2026)
- [Bring Music The Horizon: Music-Driven 360^circ Video Generation](https://mlsanity.com/paper/2607.13471/) (2026)
- [TimeCues Studio: A Workspace for Music Annotation and Algorithm Prototyping](https://mlsanity.com/paper/2609.10338/) (2026)
- [JSIR: An Audio-Aligned, Multi-Task Dataset of Jazz Standard Recordings](https://mlsanity.com/paper/2610.15356/) (2026)

## Abstract (arXiv)

Music is characterized by complex hierarchical structures. Developing a comprehensive model to capture these structures has been a significant challenge in the field of Music Information Retrieval (MIR). Prior research has mainly focused on addressing individual tasks for specific hierarchical levels, rather than providing a unified approach. In this paper, we introduce a versatile, all-in-one model that jointly performs beat and downbeat tracking as well as functional structure segmentation and labeling. The model leverages source-separated spectrograms as inputs and employs dilated neighborhood attentions to capture temporal long-term dependencies, along with non-dilated attentions for local instrumental dependencies. Consequently, the proposed model achieves state-of-the-art performance in all four tasks on the Harmonix Set while maintaining a relatively lower number of parameters compared to recent state-of-the-art models. Furthermore, our ablation study demonstrates that the concurrent learning of beats, downbeats, and segments can lead to enhanced performance, with each task mutually benefiting from the others.