# Are Large Language Models Really Robust to Word-Level Perturbations?

arXiv: https://arxiv.org/abs/2309.11166 · HTML: https://mlsanity.com/paper/2309.11166/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 37 of its 53 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://mlsanity.com/paper/1909.00277/) (2019)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)

## Built on by (most cited first, 30 of 32)

- [GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://mlsanity.com/paper/2402.19255/) (2024)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist](https://mlsanity.com/paper/2407.08733/) (2024)
- [Hijacking Large Language Models via Adversarial In-Context Learning](https://mlsanity.com/paper/2311.09948/) (2023)
- [Not All LLM Reasoners Are Created Equal](https://mlsanity.com/paper/2410.01748/) (2024)
- [Benchmarking Reasoning Robustness in Large Language Models](https://mlsanity.com/paper/2503.04550/) (2025)
- [UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models](https://mlsanity.com/paper/2502.13141/) (2025)
- [Lifelong Safety Alignment for Language Models](https://mlsanity.com/paper/2505.20259/) (2025)
- [Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrapping](https://mlsanity.com/paper/2402.07610/) (2024)
- [Measurement in the Age of LLMs: An Application to Ideological Scaling](https://mlsanity.com/paper/2312.09203/) (2023)
- [Robustness of Large Language Models to Perturbations in Text](https://mlsanity.com/paper/2407.08989/) (2024)
- [Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations](https://mlsanity.com/paper/2502.01349/) (2025)
- [QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning](https://mlsanity.com/paper/2408.10504/) (2024)
- [PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion](https://mlsanity.com/paper/2403.03788/) (2024)
- [Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors](https://mlsanity.com/paper/2510.09536/) (2025)
- [RoboDriveVLM: A Novel Benchmark and Baseline towards Robust Vision-Language Models for Autonomous Driving](https://mlsanity.com/paper/2512.01300/) (2025)
- [From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency](https://mlsanity.com/paper/2404.12145/) (2024)
- [STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models](https://mlsanity.com/paper/2502.13119/) (2025)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Exploring Robustness of Multilingual LLMs on Real-World Noisy Data](https://mlsanity.com/paper/2501.08322/) (2025)
- [Position: Stop Acting Like Language Model Agents Are Normal Agents](https://mlsanity.com/paper/2502.10420/) (2025)
- [Evaluating Robustness of Large Language Models in Enterprise Applications: Benchmarks for Perturbation Consistency Across Formats and Languages](https://mlsanity.com/paper/2601.06341/) (2026)
- [Interpreting token compositionality in LLMs: A robustness analysis](https://mlsanity.com/paper/2410.12924/) (2024)
- [Evaluating Robustness of LLMs in Question Answering on Multilingual Noisy OCR Data](https://mlsanity.com/paper/2502.16781/) (2025)
- [Semantic Needles in Document Haystacks: Sensitivity Testing of LLM-as-a-Judge Similarity Scoring](https://mlsanity.com/paper/2604.18835/) (2026)
- [Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models](https://mlsanity.com/paper/2402.17671/) (2024)
- [Where Do Prompt Perturbations Break Generation? A Segment-Level View of Robustness in LoRA-Tuned Language Models](https://mlsanity.com/paper/2605.01605/) (2026)
- [Harmonic LLMs are Trustworthy](https://mlsanity.com/paper/2404.19708/) (2024)
- [Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning](https://mlsanity.com/paper/2510.12939/) (2025)
- [Guaranteed Jailbreaking Defense via Disrupt-and-Rectify Smoothing](https://mlsanity.com/paper/2605.10582/) (2026)

## Abstract (arXiv)

The swift advancement in the scales and capabilities of Large Language Models (LLMs) positions them as promising tools for a variety of downstream tasks. In addition to the pursuit of better performance and the avoidance of violent feedback on a certain prompt, to ensure the responsibility of the LLM, much attention is drawn to the robustness of LLMs. However, existing evaluation methods mostly rely on traditional question answering datasets with predefined supervised labels, which do not align with the superior generation capabilities of contemporary LLMs. To address this issue, we propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools to evaluate the longer conversation generated from more challenging open questions by LLMs, which we refer to as the Reward Model for Reasonable Robustness Evaluation (TREvaL). Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions, a capability not entirely encompassed by individual words or letters, which may exhibit oversimplification and inherent biases. Our extensive empirical experiments demonstrate that TREvaL provides an innovative method for evaluating the robustness of an LLM. Furthermore, our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage. Notably, we are surprised to discover that robustness tends to decrease as fine-tuning (SFT and RLHF) is conducted. The code of TREval is available in https://github.com/Harry-mic/TREvaL.