# Melody or Machine: Detecting Synthetic Music with Dual-Stream Contrastive Learning arXiv: https://arxiv.org/abs/2512.00621 ยท HTML: https://mlsanity.com/paper/2512.00621/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 34 references that are in the map. - First posted on arXiv 29 Nov 2025. ## Builds on (most cited first, 24 of 24) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015) - [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021) - [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015) - [GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction](https://mlsanity.com/paper/2305.18752/) (2023) - [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016) - [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021) - [DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion](https://mlsanity.com/paper/2503.01183/) (2025) - [Audio Deepfake Detection with Self-Supervised WavLM and Multi-Fusion Attentive Classifier](https://mlsanity.com/paper/2312.08089/) (2023) - [SingFake: Singing Voice Deepfake Detection](https://mlsanity.com/paper/2309.07525/) (2023) - [SONICS: Synthetic Or Not -- Identifying Counterfeit Songs](https://mlsanity.com/paper/2408.14080/) (2024) - [CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection](https://mlsanity.com/paper/2406.02438/) (2024) - [Heterogeneity over Homogeneity: Investigating Multilingual Speech Pre-Trained Models for Detecting Audio Deepfake](https://mlsanity.com/paper/2404.00809/) (2024) - [Expertise and Dynamics within Crowdsourced Musical Knowledge Curation: A Case Study of the Genius Platform](https://mlsanity.com/paper/2006.08108/) (2020) - [X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment](https://mlsanity.com/paper/2403.11399/) (2024) - [Learning to Recognize Musical Genre from Audio](https://mlsanity.com/paper/1803.05337/) (2018) - [RefineGAN: Universally Generating Waveform Better than Ground Truth with Highly Accurate Pitch and Intensity Responses](https://mlsanity.com/paper/2111.00962/) (2021) - [Deepfake Audio Detection Using Spectrogram-based Feature and Ensemble of Deep Learning Models](https://mlsanity.com/paper/2407.01777/) (2024) - [JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images](https://mlsanity.com/paper/2409.12953/) (2024) - [MLP Singer: Towards Rapid Parallel Korean Singing Voice Synthesis](https://mlsanity.com/paper/2106.07886/) (2021) - [Investigating Prosodic Signatures via Speech Pre-Trained Models for Audio Deepfake Source Attribution](https://mlsanity.com/paper/2412.17796/) (2024) - [Generalizable Audio Deepfake Detection via Hierarchical Structure Learning and Feature Whitening in Poincar'e sphere](https://mlsanity.com/paper/2508.01897/) (2025) - [Deepfake Detection of Singing Voices With Whisper Encodings](https://mlsanity.com/paper/2501.18919/) (2025) ## Built on by (most cited first, 1 of 1) - [HAIM: Human-AI Music Datasets for AI Music Production Tracking Benchmark](https://mlsanity.com/paper/2606.01686/) (2026) ## Abstract (arXiv) The rapid evolution of end-to-end AI music generation poses an escalating threat to artistic authenticity and copyright, demanding detection methods that can keep pace. While foundational, existing models like SpecTTTra falter when faced with the diverse and rapidly advancing ecosystem of new generators, exhibiting significant performance drops on out-of-distribution (OOD) content. This generalization failure highlights a critical gap: the need for more challenging benchmarks and more robust detection architectures. To address this, we first introduce Melody or Machine (MoM), a new large-scale benchmark of over 130,000 songs (6,665 hours). MoM is the most diverse dataset to date, built with a mix of open and closed-source models and a curated OOD test set designed specifically to foster the development of truly generalizable detectors. Alongside this benchmark, we introduce CLAM, a novel dual-stream detection architecture. We hypothesize that subtle, machine-induced inconsistencies between vocal and instrumental elements, often imperceptible in a mixed signal, offer a powerful tell-tale sign of synthesis. CLAM is designed to test this hypothesis by employing two distinct pre-trained audio encoders (MERT and Wave2Vec2) to create parallel representations of the audio. These representations are fused by a learnable cross-aggregation module that models their inter-dependencies. The model is trained with a dual-loss objective: a standard binary cross-entropy loss for classification, complemented by a contrastive triplet loss which trains the model to distinguish between coherent and artificially mismatched stream pairings, enhancing its sensitivity to synthetic artifacts without presuming a simple feature alignment. CLAM establishes a new state-of-the-art in synthetic music forensics. It achieves an F1 score of 0.925 on our challenging MoM benchmark.