mlsanityData to 7 Oct 2026

LLMScan: Causal Scan for LLM Misbehavior Detection

arXiv 2410.16638 ↗First posted 22 Oct 2024cs.AI cs.CL cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Despite the success of Large Language Models (LLMs) across various fields, their potential to generate untruthful, biased and harmful responses poses significant risks, particularly in critical applications. This highlights the urgent need for systematic methods to detect and prevent such misbehavior. While existing approaches target specific issues such as harmful responses, this work introduces LLMScan, an innovative LLM monitoring technique based on causality analysis, offering a comprehensive solution. LLMScan systematically monitors the inner workings of an LLM through the lens of causal inference, operating on the premise that the LLM's `brain' behaves differently when misbehaving. By analyzing the causal contributions of the LLM's input tokens and transformer layers, LLMScan effectively detects misbehavior. Extensive experiments across various tasks and models reveal clear distinctions in the causal distributions between normal behavior and misbehavior, enabling the development of accurate, lightweight detectors for a variety of misbehavior detection tasks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 65 of its 87 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

65 of 87 references.

References · 65

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
38 more
  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  3. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  4. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  5. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  8. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  9. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  12. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  13. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  14. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  15. 2023Detecting Language Model Attacks with Perplexitycited by 411
  16. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  17. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  18. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  19. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  20. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  21. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  22. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  23. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  24. 2019Evaluating Gender Bias in Machine Translationcited by 247
  25. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  26. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  27. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 206
  28. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  29. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  30. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  31. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  32. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  33. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  34. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  35. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  36. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  37. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  38. 2024BadEdit: Backdooring large language models by model editingcited by 84

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

By area:

  1. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  2. 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 13
  3. 2025Causal-Guided Detoxify Backdoor Attack of Open-Weight LoRA Modelscited by 3
  4. 2026Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Familiescited by 2
  5. 2026Rendering Data Unlearnable by Exploiting LLM Alignment Mechanismscited by 1
  6. 2026CausalT5k: Diagnosing Refusal and Failure Modes in Trustworthy Causal Reasoning Across Causal Rungscited by 1
  7. 2026Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Modelscited by 1
  8. 2026Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Modelscited by 1
  9. 2025Microsaccade-Inspired Probing: Positional Encoding Perturbations Reveal LLM Misbehaviourscited by 0
  10. 2025SoK: a Comprehensive Causality Analysis Framework for Large Language Model Securitycited by 0
  11. 2026ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interactioncited by 0
  12. 2026I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Modelscited by 0
1 more
  1. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0