SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifolds
Not on the map. Why?
Abstract (arXiv)
Recent strides in pretrained transformer-based language models have propelled state-of-the-art performance in numerous NLP tasks. Yet, as these models grow in size and deployment, their robustness under input perturbations becomes an increasingly urgent question. Existing robustness methods often diverge between small-parameter and large-scale models (LLMs), and they typically rely on labor-intensive, sample-specific adversarial designs. In this paper, we propose a unified, local (sample-level) robustness framework (SALMAN) that evaluates model stability without modifying internal parameters or resorting to complex perturbation heuristics. Central to our approach is a novel Distance Mapping Distortion (DMD) measure, which ranks each sample's susceptibility by comparing input-to-output distance mappings in a near-linear complexity manner. By demonstrating significant gains in attack efficiency and robust training, we position our framework as a practical, model-agnostic tool for advancing the reliability of transformer-based NLP systems.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 53 of its 76 references that are in the map. First posted on arXiv 23 Aug 2025.
Builds on
53 of 76 references.
References · 53
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
38 more
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
- 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
- 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
- 2019How to Fine-Tune BERT for Text Classification?cited by 411
- 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
- 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
- 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 264
- 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
- 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
- 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
- 2020PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networkscited by 211
- 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 211
- 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
- 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 156
- 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
- 2018Learning graphs from data: A signal representation perspectivecited by 119
- 2020Reevaluating Adversarial Examples in Natural Languagecited by 85
- 2020Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERTcited by 69
- 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 56
- 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
- 2019Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brainscited by 38
- 2016Approximate Gaussian Elimination for Laplacians: Fast, Sparse, and Simplecited by 23
- 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
- 2020Manifold structure in graph embeddingscited by 17
- 2021SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluationcited by 16
- 2019DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modelingcited by 10
- 2024Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generationcited by 10
- 2024The structure of the token space for large language modelscited by 10
- 2018Graph Sparsification, Spectral Sketches, and Faster Resistance Computation, via Short Cycle Decompositionscited by 8
- 2023On Robustness of Finetuned Transformer-based NLP Modelscited by 8
- 2023Don't Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Textcited by 8
- 2022ROSE: Robust Selective Fine-tuning for Pre-trained Language Modelscited by 6
- 2021SGL: Spectral Graph Learning from Measurementscited by 5
- 2021Towards Variable-Length Textual Adversarial Attackscited by 5
- 2023PerturbScore: Connecting Discrete and Continuous Perturbations in NLPcited by 5
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.