mlsanityData to 7 Oct 2026

Onsets and Frames: Dual-Objective Piano Transcription

arXiv 1710.11153 ↗First posted 30 Oct 2017cs.SD cs.LG eess.AScited by 93

Not on the map. Why?

Abstract (arXiv)

We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and frames. Our model predicts pitch onset events and then uses those predictions to condition framewise pitch predictions. During inference, we restrict the predictions from the framewise detector by not allowing a new note to start unless the onset detector also agrees that an onset for that pitch is present in the frame. We focus on improving onsets and offsets together instead of either in isolation as we believe this correlates better with human musical perception. Our approach results in over a 100% relative improvement in note F1 score (with offsets) on the MAPS dataset. Furthermore, we extend the model to predict relative velocities of normalized audio which results in more natural-sounding transcriptions.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 21; 2023: 13; 2024: 20; 2025: 17; 2026: 11.

93 papers.

Sort:

By area:

  1. 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 109
  2. 2022HEAR: Holistic Evaluation of Audio Representationscited by 77
  3. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  4. 2022General-purpose, long-context autoregressive modeling with Perceiver ARcited by 60
  5. 2021Sequence-to-Sequence Piano Transcription with Transformerscited by 48
  6. 2021MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAEcited by 39
  7. 2021Relative Positional Encoding for Transformers with Linear Complexitycited by 36
  8. 2022Video Background Music Generation: Dataset, Method and Evaluationcited by 28
  9. 2022A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimationcited by 27
  10. 2022Unaligned Supervision For Automatic Music Transcription in The Wildcited by 25
  11. 2022Melody transcription via generative pre-trainingcited by 23
  12. 2023Automatic Piano Transcription with Hierarchical Frequency-Time Transformercited by 22
38 more
  1. 2021ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Datacited by 20
  2. 2024A Data-Driven Analysis of Robust Automatic Piano Transcriptioncited by 15
  3. 2024High Resolution Guitar Transcription via Domain Adaptationcited by 15
  4. 2024Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcriptioncited by 14
  5. 2022DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capabilitycited by 13
  6. 2024YourMT3+: Multi-instrument Music Transcription with Enhanced Transformer Architectures and Cross-dataset Stem Augmentationcited by 13
  7. 2021BERT-like Pre-training for Symbolic Piano Music Classification Taskscited by 12
  8. 2023Multitrack Music Transcription with a Time-Frequency Perceivercited by 12
  9. 2022Generating music with sentiment using Transformer-GANscited by 11
  10. 2022Pop2Piano : Pop Audio-based Piano Cover Generationcited by 10
  11. 2025Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modelingcited by 10
  12. 2024End-to-end Piano Performance-MIDI to Score Conversion with Transformerscited by 9
  13. 2021Audio-to-symbolic Arrangement via Cross-modal Music Representation Learningcited by 6
  14. 2022Jointist: Joint Learning for Multi-instrument Transcription and Its Applicationscited by 6
  15. 2024An Order-Complexity Aesthetic Assessment Model for Aesthetic-aware Music Recommendationcited by 6
  16. 2024Music Foundation Model as Generic Booster for Music Downstream Taskscited by 6
  17. 2025Unified Cross-modal Translation of Score Images, Symbolic Music, and Performance Audiocited by 6
  18. 2021Structure-Aware Audio-to-Score Alignment using Progressively Dilated Convolutional Neural Networkscited by 5
  19. 2022FretNet: Continuous-Valued Pitch Contour Streaming for Polyphonic Guitar Tablature Transcriptioncited by 5
  20. 2023A Phoneme-Informed Neural Network Model for Note-Level Singing Transcriptioncited by 5
  21. 2023REMAST: Real-time Emotion-based Music Arrangement with Soft Transitioncited by 5
  22. 2025PianoVAM: A Multimodal Piano Performance Datasetcited by 5
  23. 2021CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGANcited by 4
  24. 2022A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptionscited by 4
  25. 2022Gen'eLive! Generating Rhythm Actions in Love Live!cited by 4
  26. 2022Playing Technique Detection by Fusing Note Onset Information in Guzheng Performancecited by 4
  27. 2023Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyondcited by 4
  28. 2023MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuningcited by 4
  29. 2024Towards Efficient and Real-Time Piano Transcription Using Neural Autoregressive Modelscited by 4
  30. 2025A Comprehensive Survey on Generative AI for Video-to-Music Generationcited by 4
  31. 2025A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectivescited by 4
  32. 2022A Data-Driven Methodology for Considering Feasibility and Pairwise Likelihood in Deep Learning Based Guitar Tablature Transcription Systemscited by 3
  33. 2022An adaptive music generation architecture for games based on the deep learning Transformer modecited by 3
  34. 2022Challenges in creative generative models for music: a divergence maximization perspectivecited by 3
  35. 2023Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Trainingcited by 3
  36. 2023Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcriptioncited by 3
  37. 2024Quantifying the Corpus Bias Problem in Automatic Music Transcription Systemscited by 3
  38. 2025D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcriptioncited by 3

50 most cited of 93 All 93 (JSON)