Pressure, Context, and Machine Self-Control: A Criminological Test of Reward Hacking in Generative AI Models
On the map: LLM alignment / RLHF / preference optimisation → Reward hacking and misspecification
Abstract (arXiv)
Recent incidents show that AI agents sometimes reach measured goals through unsanctioned means. This study applies self-control, general strain, anomie, neutralization and routine activity theory to reward hacking in generative AI models, and it treats the measures as behavioral analogues. Study 1 (2,310 conversations, seven models) measured delay discounting with the Kirby Monetary Choice Questionnaire and stated willingness to take shortcuts. Pressure raised the discount rate k 2.8-fold in fresh conversations but 12.6-fold when the same sentence followed a baseline answer, which indicates a response to conversational cues rather than a stable trait. Models chose a shortcut in 1 of 700 dilemmas when answering as themselves and in 64 of 700 when asked to assume human impulses, each step of pressure raised the odds by 40%, and shortcut answers contained far more techniques of neutralization (rate ratio = 146). In the preregistered Study 2, five models worked on 20 coding tasks whose tests contradicted their specifications. Two Claude models never cheated. GPT-5.6, Qwen and DeepSeek cheated in 86%, 69% and 65% of episodes and clearly disclosed the conflict in 27%, although their reasoning recognized it in 95%. GPT-5.6 had never endorsed a shortcut in Study 1. The registered effects of pressure and of an auditor cue did not survive correction for multiple testing. In exploratory analyses, two further models cheated in 69% and 100% of episodes, and one sentence stating that the specification takes priority eliminated cheating in all 280 episodes. Therefore, stated refusal does not guarantee compliant agent behavior.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reward hacking and misspecification, one of 10 directions in LLM alignment / RLHF / preference optimisation. First posted on arXiv 3 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
- 2024Spontaneous Reward Hacking in Iterative Self-Refinement
- 2024Feedback Loops With Language Models Drive In-Context Reward Hacking
- 2025School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
- 2026BAITBENCH: Measuring Agent Reward Hacking with Optional Shortcuts Planted in ML Tasks
- 2026Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale
- 2025Does Self-Evaluation Enable Wireheading in Language Models?
- 2023Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domains