# Backpropagation Through Time For Networks With Long-Term Dependencies

arXiv: https://arxiv.org/abs/2103.15589 · HTML: https://mlsanity.com/paper/2103.15589/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 4 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Built on by (most cited first, 1 of 1)

- [Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework](https://mlsanity.com/paper/2608.08113/) (2026)

## Abstract (arXiv)

Backpropagation through time (BPTT) is a technique of updating tuned parameters within recurrent neural networks (RNNs). Several attempts at creating such an algorithm have been made including: Nth Ordered Approximations and Truncated-BPTT. These methods approximate the backpropagation gradients under the assumption that the RNN only utilises short-term dependencies. This is an acceptable assumption to make for the current state of artificial neural networks. As RNNs become more advanced, a shift towards influence by long-term dependencies is likely. Thus, a new method for backpropagation is required. We propose using the 'discrete forward sensitivity equation' and a variant of it for single and multiple interacting recurrent loops respectively. This solution is exact and also allows the network's parameters to vary between each subsequent step, however it does require the computation of a Jacobian.