# Maximizing BCI Human Feedback using Active Learning

arXiv: https://arxiv.org/abs/2008.04873 · HTML: https://mlsanity.com/paper/2008.04873/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 11 Aug 2020.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces](https://mlsanity.com/paper/1709.10163/) (2017)
- [Accelerated Robot Learning via Human Brain Signals](https://mlsanity.com/paper/1910.00682/) (2019)
- [Face valuing: Training user interfaces with facial expressions and reinforcement learning](https://mlsanity.com/paper/1606.02807/) (2016)
- [Skill Learning by Autonomous Robotic Playing using Active Learning and Creativity](https://mlsanity.com/paper/1706.08560/) (2017)

## Built on by (most cited first, 8 of 8)

- [NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities](https://mlsanity.com/paper/2311.01454/) (2023)
- [Aligning Humans and Robots via Reinforcement Learning from Implicit Human Feedback](https://mlsanity.com/paper/2507.13171/) (2025)
- [Neuro-Cognitive Reward Modeling for Human-Centered Autonomous Vehicle Control](https://mlsanity.com/paper/2603.25968/) (2026)
- [Towards Interactive Reinforcement Learning with Intrinsic Feedback](https://mlsanity.com/paper/2112.01575/) (2021)
- [Error-related Potential Variability: Exploring the Effects on Classification and Transferability](https://mlsanity.com/paper/2301.06555/) (2023)
- [Cross-Modal Consistency-Guided Active Learning for Affective BCI Systems](https://mlsanity.com/paper/2511.15138/) (2025)
- [Reinforcement Learning from Implicit Neural Feedback for Human-Aligned Robot Control](https://mlsanity.com/paper/2512.00050/) (2025)
- [An offline approach to fNIRS-guided reinforcement learning for robot behavior](https://mlsanity.com/paper/2607.14393/) (2026)

## Abstract (arXiv)

Recent advancements in textitLearning from Human Feedback present an effective way to train robot agents via inputs from non-expert humans, without a need for a specially designed reward function. However, this approach needs a human to be present and attentive during robot learning to provide evaluative feedback. In addition, the amount of feedback needed grows with the level of task difficulty and the quality of human feedback might decrease over time because of fatigue. To overcome these limitations and enable learning more robot tasks with higher complexities, there is a need to maximize the quality of expensive feedback received and reduce the amount of human cognitive involvement required. In this work, we present an approach that uses active learning to smartly choose queries for the human supervisor based on the uncertainty of the robot and effectively reduces the amount of feedback needed to learn a given task. We also use a novel multiple buffer system to improve robustness to feedback noise and guard against catastrophic forgetting as the robot learning evolves. This makes it possible to learn tasks with more complexity using lesser amounts of human feedback compared to previous methods. We demonstrate the utility of our proposed method on a robot arm reaching task where the robot learns to reach a location in 3D without colliding with obstacles. Our approach is able to learn this task faster, with less human feedback and cognitive involvement, compared to previous methods that do not use active learning.