mlsanityData to 7 Oct 2026

PerturbScore: Connecting Discrete and Continuous Perturbations in NLP

arXiv 2310.08889 ↗First posted 13 Oct 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

With the rapid development of neural network applications in NLP, model robustness problem is gaining more attention. Different from computer vision, the discrete nature of texts makes it more challenging to explore robustness in NLP. Therefore, in this paper, we aim to connect discrete perturbations with continuous perturbations, therefore we can use such connections as a bridge to help understand discrete perturbations in NLP models. Specifically, we first explore how to connect and measure the correlation between discrete perturbations and continuous perturbations. Then we design a regression task as a PerturbScore to learn the correlation automatically. Through experimental results, we find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation, surpassing previous methods used in discrete perturbation measuring. Further, the proposed PerturbScore can be well generalized to different datasets, perturbation methods, indicating that we can use it as a powerful tool to study model robustness in NLP.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 13 Oct 2023.

Builds on

22 of 29 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
10 more
  1. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  2. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  3. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  4. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  5. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  6. 2018Universal Sentence Encodercited by 485
  7. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 138
  8. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  9. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 55
  10. 2019Theoretical evidence for adversarial robustness through randomizationcited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbationcited by 10
  2. 2024Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Surveycited by 6
  3. 2024Challenges and Opportunities in Text Generation Explainabilitycited by 2
  4. 2025SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifoldscited by 1
  5. 2026Learning Perturbations to Extrapolate Your LLMcited by 0