# And what if two musical versions don't share melody, harmony, rhythm, or lyrics ?

arXiv: https://arxiv.org/abs/2210.01256 · HTML: https://mlsanity.com/paper/2210.01256/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Wav2Letter: an End-to-End ConvNet-based Speech Recognition System](https://mlsanity.com/paper/1609.03193/) (2016)
- [Towards a universal neural network encoder for time series](https://mlsanity.com/paper/1805.03908/) (2018)
- [Fully Convolutional Speech Recognition](https://mlsanity.com/paper/1812.06864/) (2018)
- [DALI: a large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigm](https://mlsanity.com/paper/1906.10606/) (2019)
- [Disentangled Multidimensional Metric Learning for Music Similarity](https://mlsanity.com/paper/2008.03720/) (2020)
- [End-to-end Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Model](https://mlsanity.com/paper/1902.06797/) (2019)
- [Audio-based Musical Version Identification: Elements and Challenges](https://mlsanity.com/paper/2109.02472/) (2021)
- [Energy Consumption of Deep Generative Audio Models](https://mlsanity.com/paper/2107.02621/) (2021)
- [Learning a Representation for Cover Song Identification Using Convolutional Neural Network](https://mlsanity.com/paper/1911.00334/) (2019)
- [Accurate and Scalable Version Identification Using Musically-Motivated Embeddings](https://mlsanity.com/paper/1910.12551/) (2019)
- [Acoustic Modeling for Automatic Lyrics-to-Audio Alignment](https://mlsanity.com/paper/1906.10369/) (2019)
- [Cover Detection using Dominant Melody Embeddings](https://mlsanity.com/paper/1907.01824/) (2019)

## Built on by (most cited first, 3 of 3)

- [Combining audio control and style transfer using latent diffusion](https://mlsanity.com/paper/2408.00196/) (2024)
- [Scalable Music Cover Retrieval Using Lyrics-Aligned Audio Embeddings](https://mlsanity.com/paper/2601.11262/) (2026)
- [SyncTrack: Rhythmic Stability and Synchronization in Multi-Track Music Generation](https://mlsanity.com/paper/2603.01101/) (2026)

## Abstract (arXiv)

Version identification (VI) has seen substantial progress over the past few years. On the one hand, the introduction of the metric learning paradigm has favored the emergence of scalable yet accurate VI systems. On the other hand, using features focusing on specific aspects of musical pieces, such as melody, harmony, or lyrics, yielded interpretable and promising performances. In this work, we build upon these recent advances and propose a metric learning-based system systematically leveraging four dimensions commonly admitted to convey musical similarity between versions: melodic line, harmonic structure, rhythmic patterns, and lyrics. We describe our deliberately simple model architecture, and we show in particular that an approximated representation of the lyrics is an efficient proxy to discriminate between versions and non-versions. We then describe how these features complement each other and yield new state-of-the-art performances on two publicly available datasets. We finally suggest that a VI system using a combination of melodic, harmonic, rhythmic and lyrics features could theoretically reach the optimal performances obtainable on these datasets.