# Rhetorical Questions in LLM Representations: A Linear Probing Study

arXiv: https://arxiv.org/abs/2604.14128 · HTML: https://mlsanity.com/paper/2604.14128/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Do LLMs write like humans? Variation in grammatical and rhetorical styles](https://mlsanity.com/paper/2410.16107/) (2024)
- [Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs](https://mlsanity.com/paper/2509.21305/) (2025)
- [Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language](https://mlsanity.com/paper/2311.04930/) (2023)
- [SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding](https://mlsanity.com/paper/2408.11319/) (2024)
- [Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection](https://mlsanity.com/paper/2412.04509/) (2024)
- [Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations](https://mlsanity.com/paper/2502.18147/) (2025)
- [Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog](https://mlsanity.com/paper/1709.05305/) (2017)
- [A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling](https://mlsanity.com/paper/2510.15081/) (2025)

## Built on by (most cited first, 3 of 3)

- [Exploring Concreteness Through a Figurative Lens](https://mlsanity.com/paper/2604.18296/) (2026)
- [Emotional Labor Strategy Preferences in LLM Personas](https://mlsanity.com/paper/2609.00310/) (2026)
- [The Linear Representation Hypothesis Needs a Group Action](https://mlsanity.com/paper/2609.27158/) (2026)

## Abstract (arXiv)

Rhetorical questions are asked not to seek information but to persuade or signal stance. How large language models internally represent them remains unclear. We analyze rhetorical questions in LLM representations using linear probes on two social-media datasets with different discourse contexts, and find that rhetorical signals emerge early and are most stably captured by last-token representations. Rhetorical questions are linearly separable from information-seeking questions within datasets, and remain detectable under cross-dataset transfer, reaching AUROC around 0.7-0.8. However, we demonstrate that transferability does not simply imply a shared representation. Probes trained on different datasets produce different rankings when applied to the same target corpus, with overlap among the top-ranked instances often below 0.2. Qualitative analysis shows that these divergences correspond to distinct rhetorical phenomena: some probes capture discourse-level rhetorical stance embedded in extended argumentation, while others emphasize localized, syntax-driven interrogative acts. Together, these findings suggest that rhetorical questions in LLM representations are encoded by multiple linear directions emphasizing different cues, rather than a single shared direction.