# An offline approach to fNIRS-guided reinforcement learning for robot behavior

arXiv: https://arxiv.org/abs/2607.14393 · HTML: https://mlsanity.com/paper/2607.14393/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces](https://mlsanity.com/paper/1709.10163/) (2017)
- [Trial without Error: Towards Safe Reinforcement Learning via Human Intervention](https://mlsanity.com/paper/1707.05173/) (2017)
- [DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback](https://mlsanity.com/paper/1810.11748/) (2018)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learning](https://mlsanity.com/paper/2007.03177/) (2020)
- [Accelerated Robot Learning via Human Brain Signals](https://mlsanity.com/paper/1910.00682/) (2019)
- [Maximizing BCI Human Feedback using Active Learning](https://mlsanity.com/paper/2008.04873/) (2020)
- [Aligning Humans and Robots via Reinforcement Learning from Implicit Human Feedback](https://mlsanity.com/paper/2507.13171/) (2025)
- [Towards Interactive Reinforcement Learning with Intrinsic Feedback](https://mlsanity.com/paper/2112.01575/) (2021)
- [Mapping Neural Signals to Agent Performance, A Step Towards Reinforcement Learning from Neural Feedback](https://mlsanity.com/paper/2506.12636/) (2025)
- [Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance](https://mlsanity.com/paper/2511.12844/) (2025)

## Abstract (arXiv)

Human-in-the-loop Reinforcement Learning has become a popular approach for training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibility of using brain signals via functional near-infrared spectroscopy (fNIRS) to modulate robot learning in simulation. We compare agents trained on passive (observational) versus active (demonstrative) interaction tasks, and test multiple methods for enhancing the RL algorithm with the neural signal, focusing on parameter augmentation in contrast to replacement. We further examine how model granularity and noise affect agent learning. Our results show that this framework is effective. The neural signal improves learning when augmenting trajectory priorities and state-action q-targets. Additionally, the framework learns successfully from offline data, offering a practical alternative for settings where real-time BCI setups are impractical or only limited data is available.