mlsanityData to 7 Oct 2026

Real-Time Recurrent Reinforcement Learning

arXiv 2311.04830 ↗First posted 8 Nov 2023cs.LG cs.NE cs.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce a biologically plausible RL framework for solving tasks in partially observable Markov decision processes (POMDPs). The proposed algorithm combines three integral parts: (1) A Meta-RL architecture, resembling the mammalian basal ganglia; (2) A biologically plausible reinforcement learning algorithm, exploiting temporal difference learning and eligibility traces to train the policy and the value-function; (3) An online automatic differentiation algorithm for computing the gradients with respect to parameters of a shared recurrent network backbone. Our experimental results show that the method is capable of solving a diverse set of partially observable reinforcement learning tasks. The algorithm we call real-time recurrent reinforcement learning (RTRRL) serves as a model of learning in biological neural networks, mimicking reward pathways in the basal ganglia.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 20 of its 50 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

20 of 50 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  6. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  7. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  8. 2019Behaviour Suite for Reinforcement Learningcited by 129
  9. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  10. 2018Differentiable plasticity: training plastic neural networks with backpropagationcited by 83
  11. 2014Random feedback weights support learning in deep neural networkscited by 47
  12. 2017Unbiased Online Recurrent Optimizationcited by 46
8 more
  1. 2019A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networkscited by 37
  2. 2020Brain-Inspired Learning on Neuromorphic Substratescited by 35
  3. 2018Approximating Real-Time Recurrent Learning with Random Kronecker Factorscited by 29
  4. 2023Online learning of long-range dependenciescited by 25
  5. 2023Exploring the Promise and Limits of Real-Time Recurrent Learningcited by 19
  6. 2022Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problemscited by 6
  7. 2019Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecturecited by 5
  8. 2018Reconciling λ-Returns with Experience Replaycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 3; 2026: 2.

6 papers.

  1. 2023Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignmentcited by 13
  2. 2025Online Fine-Tuning of Carbon Emission Predictions using Real-Time Recurrent Learning for State Space Modelscited by 1
  3. 2025Success in Humanoid Reinforcement Learning under Partial Observationcited by 0
  4. 2025TubeDAgger: Reducing the Number of Expert Interventions with Stochastic Reach-Tubescited by 0
  5. 2026Adaptive Control in Autonomous Driving via Real-Time Recurrent RLcited by 0
  6. 2026Dynamical Priors as a Training Objective in Reinforcement Learningcited by 0