mlsanityData to 14 Oct 2026

What's Pulling the Strings? Evaluating Integrity and Attribution in AI Training and Inference through Concept Shift

arXiv 2504.21042 ↗First posted 28 Apr 2025cs.CR cs.AI cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The growing adoption of artificial intelligence (AI) has amplified concerns about trustworthiness, including integrity, privacy, robustness, and bias. To assess and attribute these threats, we propose ConceptLens, a generic framework that leverages pre-trained multimodal models to identify the root causes of integrity threats by analyzing Concept Shift in probing samples. ConceptLens demonstrates strong detection performance for vanilla data poisoning attacks and uncovers vulnerabilities to bias injection, such as the generation of covert advertisements through malicious concept shifts. It identifies privacy risks in unaltered but high-risk samples, filters them before training, and provides insights into model weaknesses arising from incomplete or imbalanced training data. Additionally, at the model level, it attributes concepts that the target model is overly dependent on, identifies misleading concepts, and explains how disrupting key concepts negatively impacts the model. Furthermore, it uncovers sociological biases in generative content, revealing disparities across sociological contexts. Strikingly, ConceptLens reveals how safe training and inference data can be unintentionally and easily exploited, potentially undermining safety alignment. Our study informs actionable insights to breed trust in AI systems, thereby speeding adoption and driving greater innovation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 54 of its 86 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

54 of 86 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safety1 Apr

Other references · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  9. 2016Wide Residual Networkscited by 3,070
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  11. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  12. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
38 more
  1. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  3. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  5. 2016Modeling Context in Referring Expressionscited by 1,063
  6. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  7. 2021Membership Inference Attacks From First Principlescited by 878
  8. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  10. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  11. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  12. 2023Detecting Language Model Attacks with Perplexitycited by 431
  13. 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
  14. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
  15. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  16. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 269
  17. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  18. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 252
  19. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 246
  20. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  21. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  22. 2020Membership Leakage in Label-Only Exposurescited by 168
  23. 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 144
  24. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 122
  25. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Modelscited by 108
  26. 2023Low-Cost High-Power Membership Inference Attackscited by 107
  27. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 101
  28. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 96
  29. 2021Trustworthy AI: A Computational Perspectivecited by 93
  30. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoningcited by 92
  31. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
  32. 2020Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Driftcited by 53
  33. 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 51
  34. 2020Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solutioncited by 41
  35. 2019Deep Neural Rejection against Adversarial Examplescited by 32
  36. 2019Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderationcited by 29
  37. 2023On Hate Scaling Laws For Data-Swampscited by 28
  38. 2022On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learningcited by 26

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Grounded but Misleading: Evaluating Semantic Alignment in AI-Generated Security Explanations
  2. 2025From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMs
  3. 2026Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
  4. 2025A Peek Behind the Curtain: Using Step-Around Prompt Engineering to Identify Bias and Misinformation in GenAI Models
  5. 2026Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Text
  6. 2026Risky Business: Measuring The Faithfulness-Safety Tension
  7. 2026When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
  8. 2024Concept Arithmetics for Circumventing Concept Inhibition in Diffusion Models