mlsanityData to 14 Oct 2026

Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation

arXiv 2507.17937 ↗First posted 23 Jul 2025cs.SD cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Generative AI systems for music and video commonly use text-based filters to prevent regurgitation of copyrighted material. We expose a significant vulnerability in this approach by introducing Adversarial PhoneTic Prompting (APT), a novel attack that bypasses these safeguards by exploiting phonetic memorization--the tendency of models to bind sub-lexical acoustic patterns (phonemes, rhyme, stress, cadence) to memorized copyrighted content. APT replaces iconic lyrics with homophonic but semantically unrelated alternatives (e.g., "mom's spaghetti" becomes "Bob's confetti"), preserving phonetic structure while evading lexical filters. We evaluate APT on leading lyrics-to-song models (Suno, YuE) across English and Korean songs spanning rap, pop, and K-pop. APT achieves 91% average similarity to copyrighted originals, versus 13.7% for random lyrics and 42.2% for semantic paraphrases. Embedding analysis confirms the mechanism: YuE's text encoder treats APT-modified lyrics as near-identical to originals (cosine similarity 0.90) while Sentence-BERT semantic similarity drops to 0.71, showing the model encodes phonetic structure over meaning. This vulnerability extends cross-modally--Veo 3 reconstructs visual scenes from original music videos when prompted with APT lyrics alone, despite no visual cues in the prompt. We further show that phonetic-semantic defense signatures fail, as APT prompts exhibit higher semantic similarity than benign paraphrases. Our findings reveal that sub-lexical acoustic structure acts as a cross-modal retrieval key, rendering current copyright filters systematically vulnerable. Demo examples are available at https://jrohsc.github.io/music_attack/.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

37 of 62 references.

References · 37

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  5. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  6. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  7. 2023Simple and Controllable Music Generationcited by 488
  8. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 265
  9. 2018Music Transformercited by 211
  10. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  11. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 181
  12. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
25 more
  1. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  2. 2024Long-form music generation with latent diffusioncited by 89
  3. 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 83
  4. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generationcited by 73
  5. 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 71
  6. 2023SingSong: Generating musical accompaniments from singingcited by 55
  7. 2024VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modelingcited by 40
  8. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  9. 2023Frozen Transformers in Language Models Are Effective Visual Encoder Layerscited by 39
  10. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Modelscited by 33
  11. 2024DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generationcited by 26
  12. 2025AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluationcited by 21
  13. 2024Latent Watermarking of Audio Generative Modelscited by 16
  14. 2024Presto! Distilling Steps and Layers for Accelerating Music Generationcited by 16
  15. 2025Fast Text-to-Audio Generation with Adversarial Post-Trainingcited by 14
  16. 2024Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Modelcited by 13
  17. 2024SongCreator: Lyrics-based Universal Song Generationcited by 12
  18. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformerscited by 11
  19. 2023Generation or Replication: Auscultating Audio Latent Diffusion Modelscited by 10
  20. 2024Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audiocited by 10
  21. 2020TensorFlow Audio Models in Essentiacited by 9
  22. 2020ByteCover: Cover Song Identification via Multi-Loss Trainingcited by 9
  23. 2025Video-Guided Text-to-Music Generation Using Public Domain Movie Collectionscited by 6
  24. 2024Watermarking Training Data of Music Generation Modelscited by 2
  25. 2024CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controlscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.