Higher Order Linear Transformer
Not on the map. Why?
Abstract (arXiv)
Following up on the linear transformer part of the article from Katharopoulos et al., that takes this idea from Shen et al., the trick that produces a linear complexity for the attention mechanism is re-used and extended to a second-order approximation of the softmax normalization.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). First posted on arXiv 28 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2024: 1.
1 paper.
- 2024Linearizing Large Language Modelscited by 51
- 2024Linearizing Large Language Models10 May 2024