# Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding

arXiv: https://arxiv.org/abs/1809.03702 · HTML: https://mlsanity.com/paper/1809.03702/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2018.

## Built on by (most cited first, 30 of 34)

- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [TransDreamer: Reinforcement Learning with Transformer World Models](https://mlsanity.com/paper/2202.09481/) (2022)
- [Coordination Among Neural Modules Through a Shared Global Workspace](https://mlsanity.com/paper/2103.01197/) (2021)
- [When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment](https://mlsanity.com/paper/2307.03864/) (2023)
- [Retrieval-Augmented Reinforcement Learning](https://mlsanity.com/paper/2202.08417/) (2022)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [High-Fidelity and Freely Controllable Talking Head Video Generation](https://mlsanity.com/paper/2304.10168/) (2023)
- [Towards mental time travel: a hierarchical memory for reinforcement learning agents](https://mlsanity.com/paper/2105.14039/) (2021)
- [Fast and Slow Learning of Recurrent Independent Mechanisms](https://mlsanity.com/paper/2105.08710/) (2021)
- [A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2106.02097/) (2021)
- [Fine-Tuning Pre-trained Transformers into Decaying Fast Weights](https://mlsanity.com/paper/2210.04243/) (2022)
- [An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning](https://mlsanity.com/paper/2103.06224/) (2021)
- [Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach](https://mlsanity.com/paper/2305.18427/) (2023)
- [Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers](https://mlsanity.com/paper/2312.12340/) (2023)
- [Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis](https://mlsanity.com/paper/2306.16803/) (2023)
- [Hindsight PRIORs for Reward Learning from Human Preferences](https://mlsanity.com/paper/2404.08828/) (2024)
- [Neural Attention Models in Deep Learning: Survey and Taxonomy](https://mlsanity.com/paper/2112.05909/) (2021)
- [Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities](https://mlsanity.com/paper/2507.13158/) (2025)
- [Selective Credit Assignment](https://mlsanity.com/paper/2202.09699/) (2022)
- [Action abstractions for amortized sampling](https://mlsanity.com/paper/2410.15184/) (2024)
- [Instance Temperature Knowledge Distillation](https://mlsanity.com/paper/2407.00115/) (2024)
- [The Topological Trouble With Transformers](https://mlsanity.com/paper/2604.17121/) (2026)
- [A Framework for Inference Inspired by Human Memory Mechanisms](https://mlsanity.com/paper/2310.09297/) (2023)
- [MALT: Multi-scale Action Learning Transformer for Online Action Detection](https://mlsanity.com/paper/2405.20892/) (2024)
- [Modeling Human Behavior Part II -- Cognitive approaches and Uncertainty](https://mlsanity.com/paper/2205.06483/) (2022)
- [Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning](https://mlsanity.com/paper/2605.16318/) (2026)
- [Recoding latent sentence representations -- Dynamic gradient-based activation modification in RNNs](https://mlsanity.com/paper/2101.00674/) (2021)
- [Delayed Rewards Calibration via Reward Empirical Sufficiency](https://mlsanity.com/paper/2102.10527/) (2021)
- [Reinforcement Learning from Bagged Reward](https://mlsanity.com/paper/2402.03771/) (2024)

## Abstract (arXiv)

Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention.