Taste-aware music retrieval from audio embeddings
Not on the map. Why?
Abstract (arXiv)
Crossmodal correspondences between sound and taste are well established in psychology and neuroscience, but largely absent from content-based multimedia retrieval. We formalise taste-from-audio prediction as a content-based music information retrieval benchmark over a perceptually validated multi-source corpus, comparing ten frozen audio encoders from the four HEAR families under a shared multi-task regression head, with gated late-fusion as a configurable variant. In order to assess the effectiveness of the models, we compute absolute error and rank correlation. The strongest systems predict the five tastes within a macro RMSE of 0.134; on held-out real music their error is less than half a single rater's deviation from the consensus (RMSE 0.13 vs. 0.28), so the model tracks the group consensus more closely than an average human rater, and well below the previous state of the art baseline (0.219). On absolute error the encoders are statistically flat, with a single VGGish matching the best fusion, but gated late-fusion's advantage is confined to rank correlation (macro Pearson r 0.724 vs. 0.666). Operationalised as a content-based retrieval index, the predicted taste space ranks a 309-item pool far more faithfully than a CLAP-text baseline, which sits at chance; ridge probes and an audio-bandstop knockout read the strongest representations against documented sound-taste correspondences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
18 of 42 references.
References · 18
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2021AST: Audio Spectrogram Transformercited by 634
- 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
- 2023Simple and Controllable Music Generationcited by 480
- 2016FMA: A Dataset For Music Analysiscited by 205
- 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
- 2022HEAR: Holistic Evaluation of Audio Representationscited by 77
- 2017Current Challenges and Visions in Music Recommender Systems Researchcited by 42
6 more
- 2022Supervised and Unsupervised Learning of Audio Representations for Music Understandingcited by 34
- 2024Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challengescited by 7
- 2025OMAR-RQ: Open Music Audio Representation Model Trained with Multi-Feature Masked Token Predictioncited by 5
- 2025A Multimodal Symphony: Integrating Taste and Sound through Generative AIcited by 3
- 2025Controllable Embedding Transformation for Mood-Guided Music Retrievalcited by 1
- 2026Multimodal Dataset Normalization and Perceptual Validation for Music-Taste Correspondencescited by 1
Built on by
Not cited yet.