# LAMP: Extracting Local Decision Surfaces From Large Language Models

arXiv: https://arxiv.org/abs/2505.11772 · HTML: https://mlsanity.com/paper/2505.11772/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 17 May 2025.

## Builds on (most cited first, 27 of 27)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Do Prompt-Based Models Really Understand the Meaning of their Prompts?](https://mlsanity.com/paper/2109.01247/) (2021)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness](https://mlsanity.com/paper/2308.16175/) (2023)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [On the Origins of Linear Representations in Large Language Models](https://mlsanity.com/paper/2403.03867/) (2024)
- [On Measuring Faithfulness or Self-consistency of Natural Language Explanations](https://mlsanity.com/paper/2311.07466/) (2023)
- [DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/1906.10263/) (2019)
- [Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models](https://mlsanity.com/paper/2409.00598/) (2024)
- [Quantifying Uncertainty in Natural Language Explanations of Large Language Models](https://mlsanity.com/paper/2311.03533/) (2023)
- [LLM-Select: Feature Selection with Large Language Models](https://mlsanity.com/paper/2407.02694/) (2024)
- [Exploring Large Language Models for Feature Selection: A Data-centric Perspective](https://mlsanity.com/paper/2408.12025/) (2024)
- [GLIME: General, Stable and Local LIME Explanation](https://mlsanity.com/paper/2311.15722/) (2023)
- [HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns](https://mlsanity.com/paper/2501.16750/) (2025)
- [Wonderful Team: Zero-Shot Physical Task Planning with Visual LLMs](https://mlsanity.com/paper/2407.19094/) (2024)
- [Quantifying perturbation impacts for large language models](https://mlsanity.com/paper/2412.00868/) (2024)
- [Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding](https://mlsanity.com/paper/2501.17310/) (2025)

## Abstract (arXiv)

We introduce LAMP (Local Attribution Mapping Probe), a method that shines light onto a black-box language model's decision surface and studies how reliably a model maps its stated reasons to its reported predictions by approximating a decision surface. LAMP treats the model's own self-reported explanations as a coordinate system and fits a locally linear surrogate that links those weights to the model's output. By doing so, it reveals how much the stated factors steer the model's decisions. We apply LAMP to three tasks: sentiment analysis, controversial-topic detection, and safety-prompt auditing. Across these tasks, LAMP reveals that many language models' locally approximated linear decision landscapes overall agree with human judgments on explanation quality and, on a clinical case-file data set, align with expert assessments. Since LAMP operates without requiring access to model gradients, logits, or internal activations, it serves as a practical and lightweight framework for auditing proprietary language models, and enabling assessment of whether a model appears to behave consistently with the explanations it provides.