mlsanityData to 14 Oct 2026

Evolving choice hysteresis in reinforcement learning: comparing the adaptive value of positivity bias and gradual perseveration

arXiv 2410.19434 ↗First posted 25 Oct 2024cs.NE q-bio.NCcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

The tendency of repeating past choices more often than expected from the history of outcomes has been repeatedly empirically observed in reinforcement learning experiments. It can be explained by at least two computational processes: asymmetric update and (gradual) choice perseveration. A recent meta-analysis showed that both mechanisms are detectable in human reinforcement learning. However, while their descriptive value seems to be well established, they have not been compared regarding their possible adaptive value. In this study, we address this gap by simulating reinforcement learning agents in a variety of environments with a new variant of an evolutionary algorithm. Our results show that positivity bias (in the form of asymmetric update) is evolutionary stable in many situations, while the emergence of gradual perseveration is less systematic and robust. Overall, our results illustrate that biases can be adaptive and selected by evolution, in an environment-specific manner.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 53 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

4 of 53 references.

References · 4

  1. 2019A Survey on Practical Applications of Multi-Armed and Contextual Banditscited by 45
  2. 2024In-context learning agents are asymmetric belief updaterscited by 14
  3. 2021Optimistic Reinforcement Learning by Forward Kullback-Leibler Divergence Optimizationcited by 8
  4. 2020Improving Neural Network Learning Through Dual Variable Learning Ratescited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Weber-Fechner Law in Temporal Difference learning derived from Control as Inferencecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Evolutionary Reinforcement Learning: A Systematic Review and Future Directions
  2. 2025Evolution favours positively biased reasoning in sequential interactions with high future gains
  3. 2025Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Light
  4. 2026Continual Reinforcement Learning with Neuroevolution
  5. 2024Mutation-Bias Learning in Games
  6. 2024Relative Value Biases in Large Language Models
  7. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms
  8. 2026Evolutionary Recurrent Decision Model in Developing Adaptive and Maladaptive Behaviors