mlsanityData to 14 Oct 2026

Resilience of Large Language Models for Noisy Instructions

arXiv 2404.09754 ↗First posted 15 Apr 2024cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

As the rapidly advancing domain of natural language processing (NLP), large language models (LLMs) have emerged as powerful tools for interpreting human commands and generating text across various tasks. Nonetheless, the resilience of LLMs to handle text containing inherent errors, stemming from human interactions and collaborative systems, has not been thoroughly explored. Our study investigates the resilience of LLMs against five common types of disruptions including 1) ASR (Automatic Speech Recognition) errors, 2) OCR (Optical Character Recognition) errors, 3) grammatical mistakes, 4) typographical errors, and 5) distractive content. We aim to investigate how these models react by deliberately embedding these errors into instructions. Our findings reveal that while some LLMs show a degree of resistance to certain types of noise, their overall performance significantly suffers. This emphasizes the importance of further investigation into enhancing model resilience. In response to the observed decline in performance, our study also evaluates a "re-pass" strategy, designed to purify the instructions of noise before the LLMs process them. Our analysis indicates that correcting noisy instructions, particularly for open-source LLMs, presents significant challenges.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

25 of 50 references.

References · 25

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  8. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  9. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
13 more
  1. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  2. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  3. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  4. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  5. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 97
  6. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  7. 2020Spelling Error Correction with Soft-Masked BERTcited by 90
  8. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 57
  9. 2021Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Modelscited by 56
  10. 2020ASR Error Correction and Domain Adaptation Using Machine Translationcited by 48
  11. 2021FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognitioncited by 43
  12. 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
  13. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 7; 2026: 5.

20 papers.

Sort:

By area:

  1. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  2. 2025Enhancing LLM Robustness to Perturbed Instructions: An Empirical Studycited by 17
  3. 2025MUR: Momentum Uncertainty guided Reasoning for Large Language Modelscited by 17
  4. 2024In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Modelscited by 9
  5. 2025Evaluating Robustness of Large Language Models Against Multilingual Typographical Errorscited by 9
  6. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Modelscited by 6
  7. 2025Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmarkcited by 4
  8. 20243DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selectioncited by 3
  9. 2026MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and Benchmarkcited by 3
  10. 2024Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?cited by 2
  11. 2024Reexamining Racial Disparities in Automatic Speech Recognition Performance: The Role of Confounding by Provenancecited by 2
  12. 2024Too Big to Fool: Resisting Deception in Language Modelscited by 2
8 more
  1. 2025Fine-Tuning on Noisy Instructions: Effects on Generalization and Performancecited by 2
  2. 2026LPDS: Evaluating LLM Robustness Through Logic-Preserving Difficulty Scalingcited by 2
  3. 2026Latent Undertow: How Ordinary Typos Break Probescited by 2
  4. 2025LLM Performance for Code Generation on Noisy Taskscited by 1
  5. 2024Efficiently Computing Susceptibility to Context in Language Modelscited by 0
  6. 2025A framework for auditing grounding claimscited by 0
  7. 2026LANTERN: Language Model Assessment on Noisy and Transformed Tasks for Understanding Error and Robustness Nuancescited by 0
  8. 2026Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelitycited by 0