# PLEX: Perturbation-free Local Explanations for LLM-Based Text Classification

arXiv: https://arxiv.org/abs/2507.10596 · HTML: https://mlsanity.com/paper/2507.10596/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 53 references that are in the map.
- First posted on arXiv 12 Jul 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Visualizing and Understanding Neural Models in NLP](https://mlsanity.com/paper/1506.01066/) (2015)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [AttentionViz: A Global View of Transformer Attention](https://mlsanity.com/paper/2305.03210/) (2023)
- [Post Hoc Explanations of Language Models Can Improve Language Models](https://mlsanity.com/paper/2305.11426/) (2023)
- [TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimation](https://mlsanity.com/paper/2407.10114/) (2024)
- [Why Attentions May Not Be Interpretable?](https://mlsanity.com/paper/2006.05656/) (2020)
- [Prompting Large Language Models for Counterfactual Generation: An Empirical Study](https://mlsanity.com/paper/2305.14791/) (2023)
- [Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification](https://mlsanity.com/paper/2105.02657/) (2021)
- [Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models](https://mlsanity.com/paper/1911.06194/) (2019)
- [On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification](https://mlsanity.com/paper/2101.00196/) (2021)
- [Do Machines and Humans Focus on Similar Code? Exploring Explainability of Large Language Models in Code Summarization](https://mlsanity.com/paper/2402.14182/) (2024)
- [Structured Self-Attention Weights Encode Semantics in Sentiment Analysis](https://mlsanity.com/paper/2010.04922/) (2020)
- [A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification](https://mlsanity.com/paper/2203.07216/) (2022)

## Abstract (arXiv)

Large Language Models (LLMs) excel in text classification, but their complexity hinders interpretability, making it difficult to understand the reasoning behind their predictions. Explainable AI (XAI) methods like LIME and SHAP offer local explanations by identifying influential words, but they rely on computationally expensive perturbations. These methods typically generate thousands of perturbed sentences and perform inferences on each, incurring a substantial computational burden, especially with LLMs. To address this, we propose underlinePerturbation-free underlineLocal underlineExplanation (PLEX), a novel method that leverages the contextual embeddings extracted from the LLM and a ``Siamese network" style neural network trained to align with feature importance scores. This one-off training eliminates the need for subsequent perturbations, enabling efficient explanations for any new sentence. We demonstrate PLEX's effectiveness on four different classification tasks (sentiment, fake news, fake COVID-19 news and depression), showing more than 92% agreement with LIME and SHAP. Our evaluation using a ``stress test" reveals that PLEX accurately identifies influential words, leading to a similar decline in classification accuracy as observed with LIME and SHAP when these words are removed. Notably, in some cases, PLEX demonstrates superior performance in capturing the impact of key features. PLEX dramatically accelerates explanation, reducing time and computational overhead by two and four orders of magnitude, respectively. This work offers a promising solution for explainable LLM-based text classification.