mlsanityData to 7 Oct 2026

Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders

arXiv 2610.11938 ↗First posted 8 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial attacks on vision-language models optimize an image toward a text target, then cite the attacked model's similarity score as evidence of success. We ask whether that score - victim-space target alignment (VTS) - predicts recovery of the target by an independent model. We first build a measurement instrument: supervised judges outside the attacked geometry, real-target blend controls, shuffled-target negatives, and a reference level derived from a 50% target-image blend. Two preregistered studies then compare six contrastive encoders under a matched attack at three perturbation budgets. Robustly trained encoders (FARE, TeCoA, PMG, TRADES) transfer substantially more independent evidence than vanilla CLIP or SigLIP; all eight contrasts reject at the bootstrap floor. However, no cell reaches the blend-derived reference level. The three best cells fall within its replication band, leaving practical recovery undecided. Within robust encoders, per-sample alignment gain correlates with evidence gain (ρ = 0.24-0.51); within vanilla CLIP the correlation is consistent with zero. Across encoders we find no monotone alignment-evidence relation. VTS is therefore informative only within a fixed robust encoder, and we provide a reporting protocol in its place.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

38 of 43 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  6. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  9. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  10. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  11. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  12. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
26 more
  1. 2019Unrestricted Adversarial Examples via Semantic Manipulationcited by 84
  2. 2018Adversarial Examples that Fool both Computer Vision and Time-Limited Humanscited by 73
  3. 2025Measurement to Meaning: A Validity-Centered Framework for AI Evaluationcited by 54
  4. 2024Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustnesscited by 48
  5. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  6. 2019Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?cited by 42
  7. 2024The Double-Ellipsoid Geometry of CLIPcited by 42
  8. 2024One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Modelscited by 39
  9. 2021A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attackscited by 33
  10. 2025Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifactscited by 25
  11. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  12. 2023Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustnesscited by 20
  13. 2026A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustnesscited by 20
  14. 2022Do Perceptually Aligned Gradients Imply Adversarial Robustness?cited by 15
  15. 2021Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?cited by 12
  16. 2023CLIPAG: Towards Generator-Free Text-to-Image Generationcited by 12
  17. 2025Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metricscited by 10
  18. 2019Image Synthesis with a Single (Robust) Classifiercited by 9
  19. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  20. 2025Universal Jailbreak Suffixes Are Strong Attention Hijackerscited by 9
  21. 2026Establishing Construct Validity in LLM Capability Benchmarks Requires Nomological Networkscited by 6
  22. 2024Transcending Adversarial Perturbations: Manifold-Aided Adversarial Examples with Legitimate Semanticscited by 4
  23. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attackscited by 4
  24. 2025Concept-based Adversarial Attack: a Probabilistic Perspectivecited by 2
  25. 2025Evaluating the Evaluators: Trust in Adversarial Robustness Testscited by 2
  26. 2025SCOOTER: A Human Evaluation Framework for Unrestricted Adversarial Examplescited by 1

Built on by

Not cited yet.