mlsanityData to 14 Oct 2026

From Cognition to Computation: A Comparative Review of Human Attention and Transformer Architectures

arXiv 2407.01548 ↗First posted 25 Apr 2024q-bio.OT cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Attention is a cornerstone of human cognition that facilitates the efficient extraction of information in everyday life. Recent developments in artificial intelligence like the Transformer architecture also incorporate the idea of attention in model designs. However, despite the shared fundamental principle of selectively attending to information, human attention and the Transformer model display notable differences, particularly in their capacity constraints, attention pathways, and intentional mechanisms. Our review aims to provide a comparative analysis of these mechanisms from a cognitive-functional perspective, thereby shedding light on several open research questions. The exploration encourages interdisciplinary efforts to derive insights from human attention mechanisms in the pursuit of developing more generalized artificial intelligence.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 17 of its 63 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

17 of 63 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  8. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  9. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  10. 2021Transformers in Vision: A Surveycited by 1,013
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  12. 2019Text Summarization with Pretrained Encoderscited by 556
5 more
  1. 2015Attention-Based Models for Speech Recognitioncited by 401
  2. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  3. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  4. 2021Learning Triadic Belief Dynamics in Nonverbal Communication from Videoscited by 17
  5. 2021Modeling Communication to Coordinate Perspectives in Cooperationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 5.

6 papers.

  1. 2025Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Predictioncited by 7
  2. 2025Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the Worldcited by 2
  3. 2024Testing Components of the Attention Schema Theory in Artificial Neural Networkscited by 0
  4. 2025Intra-neuronal attention within language models Relationships between activation and semanticscited by 0
  5. 2025Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spacescited by 0
  6. 2025Morality in AI. A plea to embed morality in LLM architectures and frameworkscited by 0