mlsanityData to 14 Oct 2026

Rainbow Noise: Stress-Testing Multimodal Harmful-Meme Detectors on LGBTQ Content

arXiv 2507.19551 ↗First posted 24 Jul 2025cs.CY cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Hateful memes aimed at LGBTQ,+ communities often evade detection by tweaking either the caption, the image, or both. We build the first robustness benchmark for this setting, pairing four realistic caption attacks with three canonical image corruptions and testing all combinations on the PrideMM dataset. Two state-of-the-art detectors, MemeCLIP and MemeBLIP2, serve as case studies, and we introduce a lightweight Text Denoising Adapter (TDA) to enhance the latter's resilience. Across the grid, MemeCLIP degrades more gently, while MemeBLIP2 is particularly sensitive to the caption edits that disrupt its language processing. However, the addition of the TDA not only remedies this weakness but makes MemeBLIP2 the most robust model overall. Ablations reveal that all systems lean heavily on text, but architectural choices and pre-training data significantly impact robustness. Our benchmark exposes where current multimodal safety models crack and demonstrates that targeted, lightweight modules like the TDA offer a powerful path towards stronger defences.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 17 of its 33 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

17 of 33 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  6. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  7. 2016Universal adversarial perturbationscited by 784
  8. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  9. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  11. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 32
  12. 2024MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classificationcited by 30
5 more
  1. 2022Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shiftcited by 24
  2. 2022Robustness Analysis of Video-Language Models Against Visual and Language Perturbationscited by 23
  3. 2025MemeBLIP2: A novel lightweight multimodal system to detect harmful memescited by 8
  4. 2023HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarioscited by 7
  5. 2024Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewardscited by 10
  2. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  3. 2025Can Large Vision-Language Models Understand Multimodal Sarcasm?cited by 5
  4. 2025Lightweight Baselines for Medical Abstract Classification: DistilBERT with Cross-Entropy as a Strong Defaultcited by 3
  5. 2025Beyond More Context: Retrieval Diversity Boosts Multi-Turn Intent Understandingcited by 0
  6. 2026Physiologically Informed Deep Learning: A Multi-Scale Framework for Next-Generation PBPK Modelingcited by 0