# Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty

arXiv: https://arxiv.org/abs/2604.04182 · HTML: https://mlsanity.com/paper/2604.04182/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 5 Apr 2026.

## Builds on (most cited first, 6 of 6)

- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Consequences of Misaligned AI](https://mlsanity.com/paper/2102.03896/) (2021)
- [Deep Reinforcement Learning amidst Lifelong Non-Stationarity](https://mlsanity.com/paper/2006.10701/) (2020)
- [Adversarial Testing in LLMs: Insights into Decision-Making Vulnerabilities](https://mlsanity.com/paper/2505.13195/) (2025)
- [Rigidity in LLM Bandits with Implications for Human-AI Dyads](https://mlsanity.com/paper/2603.07717/) (2026)

## Abstract (arXiv)

Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.