ONOTE: Hypergraph-Grounded Omnimodal Reasoning for Computational Music Science
Not on the map. Why?
Abstract (arXiv)
Omnimodal notation processing, centered on sheet music, is a controlled scientific setting in which auditory, visual, symbolic, and physical representations must encode the same musical events. Yet existing work remains fragmented across recognition and transcription, rarely testing structural consistency across notation systems. Western-staff bias and underspecified model judges further conceal errors in pitch, timing, ordering, and instrument-specific constraints. We introduce ONOTE, a unified framework that treats music as a scientifically structured domain of measurable cross-representation correspondences. Its test-only benchmark draws on a diverse collection of musical sources covering staff, Jianpu, and tablature across varied genres, instruments, and structural conditions, with aligned multimodal derivatives. Four complementary tasks cover score understanding, notation conversion, audio transcription, and symbolic generation, testing pitch and duration ordering, output syntax, and disclosed instrument-specific constraints. ONOTE also constructs a provenance-bearing proposition hypergraph from external music-theory materials for entity- and hyperedge-based evidence retrieval. Deterministic validity checks, disclosed structural-compliance SMG scoring, and controlled RAG comparisons reveal gaps between visual recognition and structure-preserving outputs. Results separate perception from music-theory application and structural or physical constraint satisfaction. ONOTE provides an auditable framework for studying representation invariance and knowledge-grounded intervention in computational music science.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 91 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
16 of 91 references.
References · 16
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
- 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
- 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
- 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
- 2017Onsets and Frames: Dual-Objective Piano Transcriptioncited by 93
- 2019Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantitycited by 86
- 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
- 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
- 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 67
- 2023MuseCoco: Generating Symbolic Music from Textcited by 50
- 2023MobileDiffusion: Instant Text-to-Image Generation on Mobile Devicescited by 42
4 more
- 2022A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimationcited by 27
- 2024Generative AI Models for Different Steps in Architectural Design: A Literature Reviewcited by 12
- 2024Exploring the Design of Generative AI in Supporting Music-based Reminiscence for Older Adultscited by 4
- 2025How Generative AI supports human in conceptual designcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MuseAgent-1: Interactive Grounded Multimodal Understanding of Music Scores and Performance Audio
- 2026MuSP-Bench: Advanced Multimodal Benchmarking of Music Understanding across Score and Performance ↗
- 2025WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning
- 2025Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning
- 2025Musical Score Understanding Benchmark: Evaluating Large Language Models' Comprehension of Complete Musical Scores
- 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
- 2025MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models
- 2025The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMS