mlsanityData to 14 Oct 2026

Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation

arXiv 2007.14902 ↗First posted 29 Jul 2020cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile. Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism. Code is available at https://github.com/lironui/Linear-Attention-Mechanism.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 4; 2024: 7; 2025: 6; 2026: 1.

21 papers.

Sort:

By area:

  1. 2021Multiscale Vision Transformerscited by 897
  2. 2023Challenges and Applications of Large Language Modelscited by 254
  3. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 36
  4. 2023Why Softmax Attention Outperforms Linear Attentioncited by 15
  5. 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Studycited by 13
  6. 2022FAR: Fourier Aerial Video Recognitioncited by 10
  7. 2022FsaNet: Frequency Self-attention for Semantic Segmentationcited by 10
  8. 2023Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?cited by 10
  9. 2025ESPFormer: Doubly-Stochastic Attention with Expected Sliced Transport Planscited by 10
  10. 2023Self-Attention Amortized Distributional Projection Optimization for Sliced Wasserstein Point-Cloud Reconstructioncited by 7
  11. 2024Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Datacited by 5
  12. 2024KnowFormer: Revisiting Transformers for Knowledge Graph Reasoningcited by 5
9 more
  1. 2025Why is prompting hard? Understanding prompts on binary sequence predictorscited by 4
  2. 2025Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insightcited by 3
  3. 2024EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignmentcited by 2
  4. 2024An alternative formulation of attention pooling function in translationcited by 1
  5. 2025SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averagingcited by 1
  6. 2024DeMansia: Mamba Never Forgets Any Tokenscited by 0
  7. 2024Rank It, Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Taskscited by 0
  8. 2025Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomicscited by 0
  9. 2026MixFormer: Linear Transformer with Mixture of Memory Expertscited by 0