# Measuring Pragmatic Influence in Large Language Model Instructions

arXiv: https://arxiv.org/abs/2602.21223 · HTML: https://mlsanity.com/paper/2602.21223/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs](https://mlsanity.com/paper/2404.16873/) (2024)
- [Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://mlsanity.com/paper/2307.11760/) (2023)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models](https://mlsanity.com/paper/2403.17336/) (2024)
- [Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements](https://mlsanity.com/paper/2401.06766/) (2024)
- [Benchmarking Prompt Sensitivity in Large Language Models](https://mlsanity.com/paper/2502.06065/) (2025)
- [Control Illusion: The Failure of Instruction Hierarchies in Large Language Models](https://mlsanity.com/paper/2502.15851/) (2025)
- [Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs](https://mlsanity.com/paper/2509.01790/) (2025)
- [Emotional Manipulation Through Prompt Engineering Amplifies Disinformation Generation in AI Large Language Models](https://mlsanity.com/paper/2403.03550/) (2024)
- [FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluation](https://mlsanity.com/paper/2504.17311/) (2025)

## Built on by (most cited first, 4 of 4)

- [Rule vs.Consequence: Dissociable Internal Representations of Moral Reasoning in LLMs](https://mlsanity.com/paper/2610.16371/) (2026)
- [Imperative Interference: Social Register Shapes Instruction Topology in Large Language Models](https://mlsanity.com/paper/2603.25015/) (2026)
- [Implicit Manipulation for Skill Selection in LLM Agents with Semantic Matching](https://mlsanity.com/paper/2609.02035/) (2026)
- [Implicit Personality Representations in Humans and LLMs](https://mlsanity.com/paper/2609.12704/) (2026)

## Abstract (arXiv)

It is not only what we ask large language models (LLMs) to do that matters, but also how we ask them. Phrases like ``This is urgent'' or ``As your supervisor'' can shift model behavior without altering task content. We study this effect as pragmatic framing, contextual cues that shape directive interpretation rather than task specification. While prior work exploits such cues for prompt optimization or probes them as security vulnerabilities, pragmatic framing itself has received comparatively little attention as a target of controlled measurement in instruction following. To support its systematic study as a measurable property, we introduce a framework that combines three components: directive-framing decomposition separating framing context from task specification; a taxonomy organizing 400 instantiations of framing into 13 strategies across 4 mechanism clusters; and priority-based measurement that quantifies influence through observable shifts in directive prioritization. Evaluating five open-weight LLMs across different families and scales, we find that pragmatic framing produces systematic shifts in directive prioritization, and the effectiveness ranking of different strategies proves highly consistent across models. This reveals that susceptibility to pragmatic framing is a structured behavioral property of instruction-tuned systems. Measuring this susceptibility is a prerequisite for any deliberate response to it, and this work provides the framework to do so.