# ONOTE: Hypergraph-Grounded Omnimodal Reasoning for Computational Music Science

arXiv: https://arxiv.org/abs/2604.20719 · HTML: https://mlsanity.com/paper/2604.20719/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 91 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Mustango: Toward Controllable Text-to-Music Generation](https://mlsanity.com/paper/2311.08355/) (2023)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies](https://mlsanity.com/paper/2308.01546/) (2023)
- [Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity](https://mlsanity.com/paper/1909.08494/) (2019)
- [Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning](https://mlsanity.com/paper/2308.11276/) (2023)
- [MT3: Multi-Task Multitrack Music Transcription](https://mlsanity.com/paper/2111.03017/) (2021)
- [ChatMusician: Understanding and Generating Music Intrinsically with LLM](https://mlsanity.com/paper/2402.16153/) (2024)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [MobileDiffusion: Instant Text-to-Image Generation on Mobile Devices](https://mlsanity.com/paper/2311.16567/) (2023)
- [A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation](https://mlsanity.com/paper/2203.09893/) (2022)
- [Generative AI Models for Different Steps in Architectural Design: A Literature Review](https://mlsanity.com/paper/2404.01335/) (2024)
- [Exploring the Design of Generative AI in Supporting Music-based Reminiscence for Older Adults](https://mlsanity.com/paper/2403.01413/) (2024)
- [How Generative AI supports human in conceptual design](https://mlsanity.com/paper/2502.00283/) (2025)

## Abstract (arXiv)

Omnimodal notation processing, centered on sheet music, is a controlled scientific setting in which auditory, visual, symbolic, and physical representations must encode the same musical events. Yet existing work remains fragmented across recognition and transcription, rarely testing structural consistency across notation systems. Western-staff bias and underspecified model judges further conceal errors in pitch, timing, ordering, and instrument-specific constraints. We introduce ONOTE, a unified framework that treats music as a scientifically structured domain of measurable cross-representation correspondences. Its test-only benchmark draws on a diverse collection of musical sources covering staff, Jianpu, and tablature across varied genres, instruments, and structural conditions, with aligned multimodal derivatives. Four complementary tasks cover score understanding, notation conversion, audio transcription, and symbolic generation, testing pitch and duration ordering, output syntax, and disclosed instrument-specific constraints. ONOTE also constructs a provenance-bearing proposition hypergraph from external music-theory materials for entity- and hyperedge-based evidence retrieval. Deterministic validity checks, disclosed structural-compliance SMG scoring, and controlled RAG comparisons reveal gaps between visual recognition and structure-preserving outputs. Results separate perception from music-theory application and structural or physical constraint satisfaction. ONOTE provides an auditable framework for studying representation invariance and knowledge-grounded intervention in computational music science.