mlsanityData to 14 Oct 2026

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models

arXiv 2504.03714 ↗First posted 28 Mar 2025cs.LG cs.AI cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large Language Models (LLMs) and Vision-Language Models (VLMs) have achieved impressive performance across a wide range of tasks, yet they remain vulnerable to carefully crafted perturbations. In this study, we seek to pinpoint the sources of this fragility by identifying parameters and input dimensions (pixels or token embeddings) that are susceptible to such perturbations. To this end, we propose a stability measure called FI, First order local Influence, which is rooted in information geometry and quantifies the sensitivity of individual parameter and input dimensions. Our extensive analysis across LLMs and VLMs (from 1.5B to 13B parameters) reveals that: (I) A small subset of parameters or input dimensions with high FI values disproportionately contribute to model brittleness. (II) Mitigating the influence of these vulnerable parameters during model merging leads to improved performance.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 74 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

46 of 74 references.

References · 46

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  12. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
34 more
  1. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  4. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  7. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  9. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  10. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  11. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  12. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  13. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  14. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  15. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  16. 2023Are aligned neural networks adversarially aligned?cited by 315
  17. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
  18. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  19. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  20. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 227
  21. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 189
  22. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  23. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
  24. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 145
  25. 2023An Empirical Study of the Non-determinism of ChatGPT in Code Generationcited by 137
  26. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  27. 2018Maximal Jacobian-based Saliency Map Attackcited by 46
  28. 2024Defending Jailbreak Prompts via In-Context Adversarial Gamecited by 29
  29. 2023LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compressioncited by 26
  30. 2022CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Modelscited by 20
  31. 2023Scaling Laws for Adversarial Attacks on Language Model Activationscited by 19
  32. 2019Sensitivity Analysis of Deep Neural Networkscited by 6
  33. 2025Asymmetric Conflict and Synergy in Post-training for LLM-based Multilingual Machine Translationcited by 6
  34. 2024Stability Evaluation via Distributional Perturbation Analysiscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 61
  2. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 29
  3. 2025SciEducator: Scientific Video Understanding and Educating via Deming-Cycle Multi-Agent Systemcited by 4
  4. 2026Alignment Risks from Capability-Seeking RL Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models
  2. 2025Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update
  3. 2025Model Hemorrhage and the Robustness Limits of Large Language Models
  4. 2026Same Answer, Different Representations: Hidden instability in VLMs
  5. 2025High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models
  6. 2026COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration
  7. 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
  8. 2026Beyond Confidence: The Rhythms of Reasoning in Generative Models