mlsanityData to 7 Oct 2026

Toward Fully Self-Supervised Multi-Pitch Estimation

arXiv 2402.15569 ↗First posted 23 Feb 2024eess.AS cs.LG cs.SDcited by 5

Not on the map. Why?

Abstract (arXiv)

Multi-pitch estimation is a decades-long research problem involving the detection of pitch activity associated with concurrent musical events within multi-instrument mixtures. Supervised learning techniques have demonstrated solid performance on more narrow characterizations of the task, but suffer from limitations concerning the shortage of large-scale and diverse polyphonic music datasets with multi-pitch annotations. We present a suite of self-supervised learning objectives for multi-pitch estimation, which encourage the concentration of support around harmonics, invariance to timbral transformations, and equivariance to geometric transformations. These objectives are sufficient to train an entirely convolutional autoencoder to produce multi-pitch salience-grams directly, without any fine-tuning. Despite training exclusively on a collection of synthetic single-note audio samples, our fully self-supervised framework generalizes to polyphonic music mixtures, and achieves performance comparable to supervised models trained on conventional multi-pitch datasets.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 39 of its 99 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

39 of 99 references.

References · 39

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  3. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  4. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  5. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  6. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
  7. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  8. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  9. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
  10. 2021Contrastive Learning of Musical Representationscited by 81
  11. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  12. 2016Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applicationscited by 60
27 more
  1. 2021Sequence-to-Sequence Piano Transcription with Transformerscited by 48
  2. 2022Supervised and Unsupervised Learning of Audio Representations for Music Understandingcited by 34
  3. 2019SPICE: Self-supervised Pitch Estimationcited by 27
  4. 2022A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimationcited by 27
  5. 2022Unaligned Supervision For Automatic Music Transcription in The Wildcited by 25
  6. 2023Automatic Piano Transcription with Hierarchical Frequency-Time Transformercited by 22
  7. 2021ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Datacited by 20
  8. 2023PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objectivecited by 18
  9. 2019Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instrumentscited by 17
  10. 2023Cross-domain Neural Pitch and Periodicity Estimationcited by 16
  11. 2022Sinusoidal Frequency Estimation by Gradient Descentcited by 15
  12. 2023Multitrack Music Transcription with a Time-Frequency Perceivercited by 12
  13. 2020The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracycited by 10
  14. 2020Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervisioncited by 9
  15. 2023Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transportcited by 8
  16. 2022HPPNet: Modeling the Harmonic Structure and Pitch Invariance in Piano Transcriptioncited by 7
  17. 2022Jointist: Joint Learning for Multi-instrument Transcription and Its Applicationscited by 6
  18. 2024Singer Identity Representation Learning using Self-Supervised Techniquescited by 6
  19. 2019Learning Complex Basis Functions for Invariant Representations of Audiocited by 5
  20. 2022Unsupervised Music Source Separation Using Differentiable Parametric Source Modelscited by 4
  21. 2022HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimationcited by 4
  22. 2023Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyondcited by 4
  23. 2024On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representationscited by 4
  24. 2018Polyphonic Pitch Tracking with Deep Layered Learningcited by 3
  25. 2023Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcriptioncited by 3
  26. 2023Efficient bandwidth extension of musical signals using a differentiable harmonic plus noise modelcited by 1
  27. 2024Tempo estimation as fully self-supervised binary classificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Music102: An D12-equivariant transformer for chord progression accompanimentcited by 1
  2. 2025Investigating an Overfitting and Degeneration Phenomenon in Self-Supervised Multi-Pitch Estimationcited by 0
  3. 2025Translation-Equivariant Self-Supervised Learning for Pitch Estimation with Optimal Transportcited by 0
  4. 2026MIDI-RAE-JEPA: Hierarchical Representation Learning and Generation for Symbolic Musiccited by 0
  5. 2026CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representationscited by 0