Toxic HallucinAItions: Perturbing Prompts and Tracing LLM Circuits
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly deployed in conversational settings where user tone ranges from polite to adversarial or toxic, yet less is known about whether toxic language in otherwise semantically equivalent prompts can degrade factual reliability. We study how lexical and tone-based prompt perturbations affect the factual reliability of LLMs. Using controlled prompt variations across polite, random, and three toxicity levels, we evaluate five LLMs on ARC-Easy, GSM8K, and MMLU. We find that toxic lexical perturbations consistently reduce factual accuracy and increase uncertainty, while polite phrasing yields limited and inconsistent changes. To examine whether these answer inconsistencies correspond to internal changes, we conduct attribution-graph analyses of model activations and influences. We find that increasing toxicity selectively amplifies perturbation-sensitive variant nodes while relatively stable core reasoning nodes remain more invariant. These findings position prompt tone as a critical dimension of LLM reliability and provide behavioral and mechanistic evidence that surface-level lexical variation can alter factual outputs and internal computation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 65 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
40 of 65 references.
References · 40
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
28 more
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
- 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
- 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
- 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
- 2020Calibration of Pre-trained Transformerscited by 307
- 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
- 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 212
- 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
- 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
- 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 145
- 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 133
- 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 92
- 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 54
- 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 33
- 2024SLM-Mod: Small Language Models Surpass LLMs at Content Moderationcited by 27
- 2025Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMAcited by 23
- 2026RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactionscited by 10
- 2025Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)cited by 7
- 2026PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Supportcited by 7
- 2026AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?cited by 5
- 2026How You Ask Matters! Adaptive RAG Robustness to Query Variationscited by 3
- 2026CausalDetox: Causal Head Selection and Intervention for Language Model Detoxificationcited by 3
Built on by
Not cited yet.