# Probing Large Language Models from A Human Behavioral Perspective

arXiv: https://arxiv.org/abs/2310.05216 · HTML: https://mlsanity.com/paper/2310.05216/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation](https://mlsanity.com/paper/1912.00903/) (2019)
- [Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?](https://mlsanity.com/paper/2205.10226/) (2022)
- [Language Models Explain Word Reading Times Better Than Empirical Predictability](https://mlsanity.com/paper/2202.01128/) (2022)
- [LAST at CMCL 2021 Shared Task: Predicting Gaze Data During Reading with a Gradient Boosting Decision Tree Approach](https://mlsanity.com/paper/2104.13043/) (2021)

## Built on by (most cited first, 5 of 5)

- [How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding](https://mlsanity.com/paper/2508.20279/) (2025)
- [GazeLLM: Multimodal LLMs incorporating Human Visual Attention](https://mlsanity.com/paper/2504.00221/) (2025)
- [CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models](https://mlsanity.com/paper/2410.17714/) (2024)
- [OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses](https://mlsanity.com/paper/2503.10927/) (2025)
- [Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions](https://mlsanity.com/paper/2504.06843/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have emerged as dominant foundational models in modern NLP. However, the understanding of their prediction processes and internal mechanisms, such as feed-forward networks (FFN) and multi-head self-attention (MHSA), remains largely unexplored. In this work, we probe LLMs from a human behavioral perspective, correlating values from LLMs with eye-tracking measures, which are widely recognized as meaningful indicators of human reading patterns. Our findings reveal that LLMs exhibit a similar prediction pattern with humans but distinct from that of Shallow Language Models (SLMs). Moreover, with the escalation of LLM layers from the middle layers, the correlation coefficients also increase in FFN and MHSA, indicating that the logits within FFN increasingly encapsulate word semantics suitable for predicting tokens from the vocabulary.