# Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP

arXiv: https://arxiv.org/abs/2505.11189 · HTML: https://mlsanity.com/paper/2505.11189/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 23 of 23)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [The role of artificial intelligence in achieving the Sustainable Development Goals](https://mlsanity.com/paper/1905.00501/) (2019)
- [Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics](https://mlsanity.com/paper/2410.21272/) (2024)
- [Impossibility Theorems for Feature Attribution](https://mlsanity.com/paper/2212.11870/) (2022)
- [GLocalX -- From Local to Global Explanations of Black Box AI Models](https://mlsanity.com/paper/2101.07685/) (2021)
- [TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimation](https://mlsanity.com/paper/2407.10114/) (2024)
- [Addressing cognitive bias in medical language models](https://mlsanity.com/paper/2402.08113/) (2024)
- [Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models](https://mlsanity.com/paper/2408.03907/) (2024)
- [Rational Shapley Values](https://mlsanity.com/paper/2106.10191/) (2021)
- [Fast Interpretable Greedy-Tree Sums](https://mlsanity.com/paper/2201.11931/) (2022)
- [A semantic embedding space based on large language models for modelling human beliefs](https://mlsanity.com/paper/2408.07237/) (2024)
- [Concept-Level Explainability for Auditing & Steering LLM Responses](https://mlsanity.com/paper/2505.07610/) (2025)
- [Misinformation spreading on Facebook](https://mlsanity.com/paper/1706.09494/) (2017)
- [The power of moral words: Loaded language generates framing effects in the extreme dictator game](https://mlsanity.com/paper/1901.02314/) (2019)
- [ShapG: new feature importance method based on the Shapley value](https://mlsanity.com/paper/2407.00506/) (2024)

## Built on by (most cited first, 3 of 3)

- [Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation](https://mlsanity.com/paper/2605.03058/) (2026)
- [Assessing Model-Agnostic XAI Methods against EU AI Act Explainability Requirements](https://mlsanity.com/paper/2604.09628/) (2026)
- [Security and Privacy in Agentic AI: Grand Challenges and Future Directions](https://mlsanity.com/paper/2607.06608/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can amplify misinformation, undermining societal goals such as the UN SDGs. We study three documented drivers of misinformation (valence framing, information overload, and oversimplification) often shaped by default beliefs. Building on evidence that LLMs encode such defaults (e.g., "joy is positive", "math is complex") and can act as "bags of heuristics", we ask whether belief-driven heuristics behind misinformation-related behaviour can be recovered from black-box LLM behaviour as explicit rules. A key obstacle is that global rule-extraction methods in explainable AI (XAI) are built for numerical input-output data, not text. We address this by eliciting global LLM beliefs and mapping them to numerical scores via statistically validated abstractions, enabling off-the-shelf global XAI to detect belief-driven heuristics. For ground truth, we inject nonlinear behavioural triggers of increasing complexity (univariate, conjunctive, non-convex) into GPT-family and Llama models via system instructions. We find that RuleFit often misses non-univariate triggers, while global SHAP better ranks conjunctive trigger features but yields no symbolic rules. To bridge this gap, we propose RuleSHAP, a rule-extraction algorithm that couples global SHAP aggregates with rule induction to better capture non-univariate triggers, improving MRR@1 over RuleFit by +82% on average. Our results suggest a practical pathway for surfacing behavioural triggers in LLMs.