# Gated Feedback Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1502.02367 · HTML: https://mlsanity.com/paper/1502.02367/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2015.

## Built on by (most cited first, 30 of 54)

- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agents](https://mlsanity.com/paper/2307.07871/) (2023)
- [Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances](https://mlsanity.com/paper/2209.13131/) (2022)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Toward Understanding In-context vs. In-weight Learning](https://mlsanity.com/paper/2410.23042/) (2024)
- [PAM: Understanding Product Images in Cross Product Category Attribute Extraction](https://mlsanity.com/paper/2106.04630/) (2021)
- [A novel time-frequency Transformer based on self-attention mechanism and its application in fault diagnosis of rolling bearings](https://mlsanity.com/paper/2104.09079/) (2021)
- [How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies](https://mlsanity.com/paper/2110.07034/) (2021)
- [SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors](https://mlsanity.com/paper/2606.23444/) (2026)
- [Unsupervised Learning of Sampling Distributions for Particle Filters](https://mlsanity.com/paper/2302.01174/) (2023)
- [Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data](https://mlsanity.com/paper/2305.12011/) (2023)
- [CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale](https://mlsanity.com/paper/2502.16645/) (2025)
- [SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2107.00956/) (2021)
- [Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey](https://mlsanity.com/paper/2402.07249/) (2024)
- [Mamba-VA: A Mamba-based Approach for Continuous Emotion Recognition in Valence-Arousal Space](https://mlsanity.com/paper/2503.10104/) (2025)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Graph augmented Deep Reinforcement Learning in the GameRLand3D environment](https://mlsanity.com/paper/2112.11731/) (2021)
- [Godot Reinforcement Learning Agents](https://mlsanity.com/paper/2112.03636/) (2021)
- [Active hypothesis testing in unknown environments using recurrent neural networks and model free reinforcement learning](https://mlsanity.com/paper/2303.10623/) (2023)
- [Object-oriented backdoor attack against image captioning](https://mlsanity.com/paper/2401.02600/) (2024)
- [DBTagger: Multi-Task Learning for Keyword Mapping in NLIDBs Using Bi-Directional Recurrent Neural Networks](https://mlsanity.com/paper/2101.04226/) (2021)
- [Adaptive Approach Phase Guidance for a Hypersonic Glider via Reinforcement Meta Learning](https://mlsanity.com/paper/2107.14764/) (2021)
- [Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation](https://mlsanity.com/paper/2203.03078/) (2022)
- [Learning in Feedback-driven Recurrent Spiking Neural Networks using full-FORCE Training](https://mlsanity.com/paper/2205.13585/) (2022)
- [Multi-Objective Evolutionary Neural Architecture Search for Recurrent Neural Networks](https://mlsanity.com/paper/2403.11173/) (2024)
- [Deep Learning Calabi-Yau four folds with hybrid and recurrent neural network architectures](https://mlsanity.com/paper/2405.17406/) (2024)
- [SocialAI 0.1: Towards a Benchmark to Stimulate Research on Socio-Cognitive Abilities in Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2104.13207/) (2021)
- [Piecewise Linear Units Improve Deep Neural Networks](https://mlsanity.com/paper/2108.00700/) (2021)
- [Integrated Guidance and Control for Lunar Landing using a Stabilized Seeker](https://mlsanity.com/paper/2112.08540/) (2021)
- [xDBTagger: Explainable Natural Language Interface to Databases Using Keyword Mappings and Schema Graph](https://mlsanity.com/paper/2210.03768/) (2022)

## Abstract (arXiv)

In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pair of layers. The recurrent signals exchanged between layers are gated adaptively based on the previous hidden states and the current input. We evaluated the proposed GF-RNN with different types of recurrent units, such as tanh, long short-term memory and gated recurrent units, on the tasks of character-level language modeling and Python program evaluation. Our empirical evaluation of different RNN units, revealed that in both tasks, the GF-RNN outperforms the conventional approaches to build deep stacked RNNs. We suggest that the improvement arises because the GF-RNN can adaptively assign different layers to different timescales and layer-to-layer interactions (including the top-down ones which are not usually present in a stacked RNN) by learning to gate these interactions.