# How Persuasive is Your Context?

arXiv: https://arxiv.org/abs/2509.17879 · HTML: https://mlsanity.com/paper/2509.17879/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence](https://mlsanity.com/paper/2210.13701/) (2022)
- [Automated Annotation with Generative AI Requires Validation](https://mlsanity.com/paper/2306.00176/) (2023)
- [Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge](https://mlsanity.com/paper/2305.01651/) (2023)
- [Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation](https://mlsanity.com/paper/2509.08825/) (2025)
- [Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning](https://mlsanity.com/paper/2305.11383/) (2023)
- [Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts](https://mlsanity.com/paper/2407.10747/) (2024)
- [Controllable Context Sensitivity and the Knob Behind It](https://mlsanity.com/paper/2411.07404/) (2024)
- [Prompt Stability Scoring for Text Annotation with Large Language Models](https://mlsanity.com/paper/2407.02039/) (2024)
- [Towards Coding Social Science Datasets with Language Models](https://mlsanity.com/paper/2306.02177/) (2023)
- [Are Chatbots Reliable Text Annotators? Sometimes](https://mlsanity.com/paper/2311.05769/) (2023)
- [Measuring Scalar Constructs in Social Science with LLMs](https://mlsanity.com/paper/2509.03116/) (2025)
- [Language Models over Canonical Byte-Pair Encodings](https://mlsanity.com/paper/2506.07956/) (2025)

## Built on by (most cited first, 1 of 1)

- [AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs](https://mlsanity.com/paper/2608.14320/) (2026)

## Abstract (arXiv)

Two central capabilities of language models (LMs) are: (i) drawing on prior knowledge about entities, which allows them to answer queries such as "What's the official language of Austria?", and (ii) adapting to new information provided in context, e.g., "Pretend the official language of Austria is Tagalog.", that is pre-pended to the question. In this article, we introduce targeted persuasion score (TPS), designed to quantify how persuasive a given context is to an LM where persuasion is operationalized as the ability of the context to alter the LM's answer to the question. In contrast to evaluating persuasiveness only by inspecting the greedily decoded answer under the model, TPS provides a more fine-grained view of model behavior. Based on the Wasserstein distance, TPS measures how much a context shifts a model's original answer distribution toward a target distribution. Empirically, through a series of experiments, we show that TPS captures a more nuanced notion of persuasiveness than previously proposed metrics.