# Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance

arXiv: https://arxiv.org/abs/2511.12844 · HTML: https://mlsanity.com/paper/2511.12844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)
- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 8 of 8)

- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback](https://mlsanity.com/paper/1810.11748/) (2018)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learning](https://mlsanity.com/paper/2206.13316/) (2022)
- [Deep Learning in fNIRS: A review](https://mlsanity.com/paper/2201.13371/) (2022)
- [Face valuing: Training user interfaces with facial expressions and reinforcement learning](https://mlsanity.com/paper/1606.02807/) (2016)
- [From "Thumbs Up" to "10 out of 10": Reconsidering Scalar Feedback in Interactive Reinforcement Learning](https://mlsanity.com/paper/2311.10284/) (2023)
- [Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2405.00746/) (2024)

## Built on by (most cited first, 1 of 1)

- [An offline approach to fNIRS-guided reinforcement learning for robot behavior](https://mlsanity.com/paper/2607.14393/) (2026)

## Abstract (arXiv)

Reinforcement Learning from Human Feedback (RLHF) is a methodology that aligns agent behavior with human preferences by integrating user feedback into the agent's training process. This paper introduces a framework that guides agent training through implicit neural signals, with a focus on the neural classification problem. Our work presents and releases a novel dataset of functional near-infrared spectroscopy (fNIRS) recordings collected from 25 human participants across three domains: Pick-and-Place Robot, Lunar Lander, and Flappy Bird. We train multiple classifiers to predict varying levels of agent performance (optimal, suboptimal, or worst-case) from windows of preprocessed fNIRS features, achieving an average F1 score of 67% for binary and 46% for multi-class classification across conditions and domains. We also train multiple regressors to predict the degree of deviation between an agent's chosen action and a set of near-optimal policy actions, providing a continuous measure of performance. Finally, we evaluate cross-subject generalization and show that fine-tuning pre-trained models with a small sample of subject-specific data increases average F1 scores by 17% and 41% for binary and multi-class models, respectively. Our results demonstrate that mapping implicit fNIRS signals to agent performance is feasible and can be improved, laying the foundation for future Reinforcement Learning from Neural Feedback (RLNF) systems.