# Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation

arXiv: https://arxiv.org/abs/2603.18893 · HTML: https://mlsanity.com/paper/2603.18893/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability.
- Builds on 51 of its 70 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Placed in

- Mechanistic interpretability / [Probing deception, bias and beliefs](https://mlsanity.com/mechinterp/probing-deception-bias-and-beliefs/)

## Builds on (most cited first, 30 of 51)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [Do Large Language Models Know What They Don't Know?](https://mlsanity.com/paper/2305.18153/) (2023)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [Looking Inward: Language Models Can Learn About Themselves by Introspection](https://mlsanity.com/paper/2410.13787/) (2024)
- [The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models](https://mlsanity.com/paper/2601.10387/) (2026)
- [Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?](https://mlsanity.com/paper/2405.16908/) (2024)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [Style Vectors for Steering Generative Large Language Model](https://mlsanity.com/paper/2402.01618/) (2024)
- [Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench](https://mlsanity.com/paper/2308.03656/) (2023)
- [Measuring and Controlling Instruction (In)Stability in Language Model Dialogs](https://mlsanity.com/paper/2402.10962/) (2024)
- [EmoBench: Evaluating the Emotional Intelligence of Large Language Models](https://mlsanity.com/paper/2402.12071/) (2024)
- [Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations](https://mlsanity.com/paper/2505.13763/) (2025)
- [Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2511.00222/) (2025)

## Built on by (most cited first, 1 of 1)

- [Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation](https://mlsanity.com/paper/2608.20569/) (2026)

## Abstract (arXiv)

Language models are often asked to rate their internal states with a number, but it is unclear how reliable this measure is. We train probes for four affective concepts (wellbeing, interest, focus and impulsivity) in eight instruction-tuned models to track these states over multi-turn conversations. We compare each probe's score with a numeric self-report of the same state, elicited at every turn. We find that rating questions displace the probed state, and that the rating is only weakly coupled with the probe score before the question but tracks the post-question probe much more strongly. Steering experiments suggest that models use the affective directions to inform ratings, but that relationship is not specific: ratings are associated with other concepts' probes as much as with their own, and change little when the question is about the user. Finally, neither the pre-question coupling nor concept specificity improves with scale (1B-27B). We warn about the effect a question can have on the internal states one is asking about, and caution against numeric self-report as a reliable measure of those states.