# A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry

arXiv: https://arxiv.org/abs/1906.11286 · HTML: https://mlsanity.com/paper/1906.11286/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [Deep Annotation of Therapeutic Working Alliance in Psychotherapy](https://mlsanity.com/paper/2204.05522/) (2022)
- [Neural Topic Modeling of Psychotherapy Sessions](https://mlsanity.com/paper/2204.10189/) (2022)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Working Alliance Transformer for Psychotherapy Dialogue Classification](https://mlsanity.com/paper/2210.15603/) (2022)
- [SupervisorBot: NLP-Annotated Real-Time Recommendations of Psychotherapy Treatment Strategies with Deep Reinforcement Learning](https://mlsanity.com/paper/2208.13077/) (2022)
- [Survey: Multi-Armed Bandits Meet Large Language Models](https://mlsanity.com/paper/2505.13355/) (2025)
- [Psychotherapy AI Companion with Reinforcement Learning Recommendations and Interpretable Policy Dynamics](https://mlsanity.com/paper/2303.09601/) (2023)
- [Computational Inference in Cognitive Science: Operational, Societal and Ethical Considerations](https://mlsanity.com/paper/2210.13526/) (2022)
- [TherapyView: Visualizing Therapy Sessions with Temporal Topic Modeling and AI-Generated Arts](https://mlsanity.com/paper/2302.10845/) (2023)
- [Reward-Punishment Reinforcement Learning with Maximum Entropy](https://mlsanity.com/paper/2405.11784/) (2024)
- [Assessing AI Utility: The Random Guesser Test for Sequential Decision-Making Systems](https://mlsanity.com/paper/2407.20276/) (2024)
- [Etat de l'art sur l'application des bandits multi-bras](https://mlsanity.com/paper/2101.00001/) (2021)
- [Regularized Reward-Punishment Reinforcement Learning](https://mlsanity.com/paper/2606.28152/) (2026)

## Abstract (arXiv)

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.