mlsanityData to 14 Oct 2026

Backdoor Learning in Language Models and Vision-Language Models

arXiv 2608.18095 ↗First posted 8 Jun 2026cs.CL cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements come with increased vulnerabilities, notably through backdoor attacks that pose severe security threats. This thesis addresses two critical dimensions of Trustworthy AI and Efficient Multimodal Representation Learning: (1) security through analyzing, detecting, and designing backdoor attacks in NLP and VLMs, and (2) efficiency through advanced multimodal representation methods tailored for clinical and medical imaging applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 61 of its 262 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

61 of 262 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
  2. 2024Test-Time Backdoor Attacks on Multimodal Large Language Models13 Feb
  3. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb

Other references · 58

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  11. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  12. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
38 more
  1. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  2. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 263
  3. 2021Poisoning and Backdooring Contrastive Learningcited by 151
  4. 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 103
  5. 2021Triggerless Backdoor Attack for NLP Tasks with Clean Labelscited by 66
  6. 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 63
  7. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 60
  8. 2021Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attackscited by 49
  9. 2021An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defencescited by 42
  10. 2019Using Clinical Notes with Time Series Data for ICU Managementcited by 38
  11. 2023Knowledge Distillation on Graphs: A Surveycited by 38
  12. 2024Physical Backdoor Attack can Jeopardize Driving with Vision-Large-Language Modelscited by 33
  13. 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaborationcited by 28
  14. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 28
  15. 2020ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generatorscited by 27
  16. 2021How to Leverage Multimodal EHR Data for Better Medical Predictions?cited by 26
  17. 2023LEMON: Lossless model expansioncited by 23
  18. 2024Task-Agnostic Detector for Insertion-Based Backdoor Attackscited by 23
  19. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  20. 2024The Landscape and Challenges of HPC Research and LLMscited by 22
  21. 2025Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agentscited by 22
  22. 2021Predicting Patient Outcomes with Graph Representation Learningcited by 21
  23. 2024Unlocking Memorization in Large Language Models with Dynamic Soft Promptingcited by 21
  24. 2024Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compressioncited by 18
  25. 2022A Survey of Neural Trojan Attacks and Defenses in Deep Learningcited by 17
  26. 2024BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learningcited by 16
  27. 2021On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformerscited by 14
  28. 2024Recent Advances in Multimodal Affective Computing: An NLP Perspectivecited by 13
  29. 2024Machine-Generated Text Localizationcited by 11
  30. 2019S2ORC: The Semantic Scholar Open Research Corpuscited by 10
  31. 2024Text Sentiment Analysis and Classification Based on Bidirectional Gated Recurrent Units (GRUs) Modelcited by 10
  32. 2023Instances Need More Care: Rewriting Prompts for Instances with LLMs in the Loop Yields Better Zero-Shot Performancecited by 9
  33. 2024Advances in Multiple Instance Learning for Whole Slide Image Analysis: Techniques, Challenges, and Future Directionscited by 7
  34. 2022Attention Hijacking in Trojan Transformerscited by 6
  35. 2025Efficient Whole Slide Pathology VQA via Token Compressioncited by 6
  36. 2023Comparative Analysis of CPU and GPU Profiling for Deep Learning Modelscited by 4
  37. 2024Fortran2CPP: Automating Fortran-to-C++ Translation using LLMs via Multi-Turn Dialogue and Dual-Agent Integrationcited by 4
  38. 2023Graphene: Infrastructure Security Posture Analysis with AI-generated Attack Graphscited by 3

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models
  2. 2024Backdooring Vision-Language Models with Out-Of-Distribution Data
  3. 2024TrojVLM: Backdoor Attack Against Vision Language Models
  4. 2025Concept-Guided Backdoor Attack on Vision Language Models
  5. 2026Phantasia: Context-Adaptive Backdoors in Vision Language Models
  6. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security
  7. 2026Breaking the weakest link to evade vision language models
  8. 2026VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts