# The Internal Anatomy of Strategic Choice in Large Language Models

arXiv: https://arxiv.org/abs/2609.07478 · HTML: https://mlsanity.com/paper/2609.07478/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 7 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Emotion Concepts and their Function in a Large Language Model](https://mlsanity.com/paper/2604.07729/) (2026)
- [Densing Law of LLMs](https://mlsanity.com/paper/2412.04315/) (2024)
- [Six Fallacies in Substituting Large Language Models for Human Participants](https://mlsanity.com/paper/2402.04470/) (2024)
- [The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?](https://mlsanity.com/paper/2305.07970/) (2023)
- [Capturing the Complexity of Human Strategic Decision-Making with Machine Learning](https://mlsanity.com/paper/2408.07865/) (2024)
- [Predicting Human Behavior in Unrepeated, Simultaneous-Move Games](https://mlsanity.com/paper/1306.0918/) (2013)
- [Behavioral Economics of AI: LLM Biases and Corrections](https://mlsanity.com/paper/2602.09362/) (2026)
- [On the Non-Identifiability of Steering Vectors in Large Language Models](https://mlsanity.com/paper/2602.06801/) (2026)
- [Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations](https://mlsanity.com/paper/2505.11615/) (2025)
- [Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions](https://mlsanity.com/paper/2605.00226/) (2026)
- [Representation Without Control: Testing the Realization Effect in Language Models](https://mlsanity.com/paper/2605.25151/) (2026)
- [A game theory for foundation models shows new paths to rational cooperation through similarity inference](https://mlsanity.com/paper/2608.03958/) (2026)

## Abstract (arXiv)

Large language models act as strategic agents and models of human choice, yet choosing like a strategic agent does not mean computing like one. We recorded activations from four open-weight models --- dense and mixture-of-experts, including a matched base--instruct pair --- in one-shot play of 144 strict ordinal 2×2 games. We followed a prespecified incentive from prompt, through activations, to choice. Dense models mirrored the unadjusted human decline with game complexity. Incentive and choice were detectable in every model, but models differed in whether incentive reached the choice, aligned with it and, where tested, whether strengthening it shifted preference. The base and instruction-tuned Qwen2.5 models chose almost identically at baseline yet differed in whether incentive reached choice. Fixed decision cues were distinguishable internally but changed choices selectively. Similar behaviour can rest on different computation; post-training can reshape the path from represented incentive to decision while leaving behaviour and decodable information largely intact.