# Balancing Privacy and Efficiency: Music Information Retrieval via Additive Homomorphic Encryption

arXiv: https://arxiv.org/abs/2508.07044 · HTML: https://mlsanity.com/paper/2508.07044/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 80 references that are in the map.
- First posted on arXiv 9 Aug 2025.

## Builds on (most cited first, 23 of 23)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [TenSEAL: A Library for Encrypted Tensor Operations Using Homomorphic Encryption](https://mlsanity.com/paper/2104.03152/) (2021)
- [Deep Learning Techniques for Music Generation -- A Survey](https://mlsanity.com/paper/1709.01620/) (2017)
- [PiRhDy: Learning Pitch-, Rhythm-, and Dynamics-aware Embeddings for Symbolic Music](https://mlsanity.com/paper/2010.08091/) (2020)
- [SANNS: Scaling Up Secure Approximate k-Nearest Neighbors Search](https://mlsanity.com/paper/1904.02033/) (2019)
- [Accurate and Scalable Version Identification Using Musically-Motivated Embeddings](https://mlsanity.com/paper/1910.12551/) (2019)
- [Secure k-Nearest Neighbor Query over Encrypted Data in Outsourced Environments](https://mlsanity.com/paper/1307.4824/) (2013)
- [Prism: Private Verifiable Set Computation over Multi-Owner Outsourced Databases](https://mlsanity.com/paper/2104.03354/) (2021)
- [A Note on Efficient Privacy-Preserving Similarity Search for Encrypted Vectors](https://mlsanity.com/paper/2502.14291/) (2025)

## Abstract (arXiv)

Modern music retrieval runs on vector embeddings, and once these embeddings are shared for search or matching they can be copied, probed, or used to train generative models. Fully homomorphic encryption can compute on them but is impractical at scale because of ciphertext--ciphertext multiplication and bootstrapping. We observe that when only one operand is encrypted, the query or the database, similarity search reduces to ciphertext addition and ciphertext--plaintext multiplication, which additive homomorphic encryption supports cheaply (through Paillier, or CKKS restricted to additive operations). Building on this observation, we (i) implement two music-specific inference attacks and quantify the privacy--utility tradeoff of provable mitigations, (ii) introduce structure-aware additive primitives with learned per-block weighting at no extra cryptographic cost, and (iii) show across four audio datasets that additive search preserves nearest-neighbor rankings exactly while avoiding the ciphertext--ciphertext multiplication and bootstrapping of full-depth FHE and scaling far better in embedding dimension than a true-additive Paillier baseline.