mlsanityData to 7 Oct 2026

Backprop-Free Reinforcement Learning with Active Neural Generative Coding

arXiv 2107.07046 ↗First posted 10 Jul 2021cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several simple control problems that our framework performs competitively with deep Q-learning. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 105 references that are in the map. First posted on arXiv 10 Jul 2021.

Builds on

36 of 105 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  7. 2018Exploration by Random Network Distillationcited by 966
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  11. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  12. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
24 more
  1. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
  2. 2019Efficient Exploration via State Marginal Matchingcited by 186
  3. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
  4. 2014Difference Target Propagationcited by 144
  5. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  6. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 135
  7. 2018Model-Based Active Explorationcited by 105
  8. 2018Iterative Amortized Inferencecited by 93
  9. 2017The Uncertainty Bellman Equation and Explorationcited by 81
  10. 2016Towards deep learning with segregated dendritescited by 71
  11. 2015Towards Biologically Plausible Deep Learningcited by 66
  12. 2019Deep Active Inference as Variational Policy Gradientscited by 52
  13. 2020Reinforcement Learning through Active Inferencecited by 47
  14. 2020Meta-Learning through Hebbian Plasticity in Random Networkscited by 47
  15. 2017Deep Active Inferencecited by 44
  16. 2020The Neural Coding Framework for Learning Generative Modelscited by 44
  17. 2018Biologically Motivated Algorithms for Propagating Local Target Representationscited by 43
  18. 2018Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representationscited by 32
  19. 2019Scaling active inferencecited by 30
  20. 2019A Unified Bellman Optimality Principle Combining Reward Maximization and Empowermentcited by 15
  21. 2017Learning to Adapt by Minimizing Discrepancycited by 12
  22. 2018Computational Theories of Curiosity-Driven Learningcited by 11
  23. 2021Prior Preference Learning from Experts:Designing a Reward with Active Inferencecited by 8
  24. 2017Training Language Models Using Target-Propagationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 2; 2025: 1; 2026: 1.

12 papers.

  1. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  2. 2022Predictive Coding beyond Gaussian Distributionscited by 16
  3. 2022Is Bio-Inspired Learning Better than Backprop? Benchmarking Bio Learning vs. Backpropcited by 9
  4. 2024Predictive Coding Networks and Inference Learning: Tutorial and Surveycited by 7
  5. 2022Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problemscited by 6
  6. 2023Active Sensing with Predictive Coding and Uncertainty Minimizationcited by 5
  7. 2022Maze Learning using a Hyperdimensional Predictive Processing Cognitive Architecturecited by 4
  8. 2024Temporal-Difference Learning Using Distributed Error Signalscited by 3
  9. 2023A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimizationcited by 2
  10. 2023Active Inference in Hebbian Learning Networkscited by 0
  11. 2025Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learningcited by 0
  12. 2026Optimizing Neurorobot Policy under Limited Demonstration Data through Preference Regretcited by 0