mlsanityData to 14 Oct 2026

FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustness

arXiv 2605.08896 ↗First posted 9 May 2026cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Robust adaptation of LLMs and VLMs is often evaluated by average accuracy or average consistency under perturbations. However, these averages can hide a structured failure mode: a prediction may remain correct while probability mass already flows from particular true classes toward systematic wrong competitors near the decision boundary. In this paper, we formalize this phenomenon as margin-aware error flow and introduce FragileFlow, a plug-in regularizer that uses a calibrated margin buffer to identify correct-but-fragile predictions and organize their off-class probability mass into a class-wise vulnerable-risk matrix. Theoretically, we provide the first PAC-Bayes upper bound for this margin-aware error-flow object, showing how empirical spectral control yields a conservative route to deterministic worst-class robustness under a stability condition. Experiments on multiple-choice LLM benchmarks and few-shot CLIP adaptation show that FragileFlow consistently improves the proposed theory-facing risk measures over matched baselines, yields perturbed worst-class accuracy gains in most settings, and preserves clean accuracy across comparisons.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

41 of 51 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2013Describing Textures in the Wildcited by 2,125
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  11. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  12. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
29 more
  1. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  2. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 264
  3. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 254
  4. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  5. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  6. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 157
  7. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  8. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 120
  9. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  10. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 76
  11. 2023CFA: Class-wise Calibrated Fair Adversarial Trainingcited by 53
  12. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  13. 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
  14. 2020Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracycited by 45
  15. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  16. 2021Analysis and Applications of Class-wise Robustness in Adversarial Trainingcited by 33
  17. 2023WAT: Improve the Worst-class Robustness in Adversarial Trainingcited by 27
  18. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
  19. 2024TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Modelscited by 23
  20. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 19
  21. 2022Improving Robust Fairness via Balance Adversarial Trainingcited by 17
  22. 2025Enhancing LLM Robustness to Perturbed Instructions: An Empirical Studycited by 17
  23. 2025Enhancing Robust Fairness via Confusional Spectral Regularizationcited by 11
  24. 2023PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descentcited by 10
  25. 2025Few-Shot Adversarial Low-Rank Fine-Tuning of Vision-Language Modelscited by 9
  26. 2025Tapas Are Free! Training-Free Adaptation of Programmatic Agents via LLM-Guided Program Synthesis in Dynamic Environmentscited by 9
  27. 2022Improving Robust Generalization by Direct PAC-Bayesian Bound Minimizationcited by 6
  28. 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Setscited by 6
  29. 2026Where Do Prompt Perturbations Break Generation? A Segment-Level View of Robustness in LoRA-Tuned Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflictscited by 0
  2. 2026Do We Need Complex Topology Control? Distinct-Peer Random Routing Improves Cost-Efficiency in Sparse Multi-Agent Debatecited by 0
  3. 2026Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancingcited by 0