mlsanityData to 14 Oct 2026

Serial Position Effects of Large Language Models

arXiv 2406.15981 ↗First posted 23 Jun 2024cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable capabilities in zero-shot learning applications, generating responses to queries using only pre-training information without the need for additional fine-tuning. This represents a significant departure from traditional machine learning approaches. Previous research has indicated that LLMs may exhibit serial position effects, such as primacy and recency biases, which are well-documented cognitive biases in human psychology. Our extensive testing across various tasks and models confirms the widespread occurrence of these effects, although their intensity varies. We also discovered that while carefully designed prompts can somewhat mitigate these biases, their effectiveness is inconsistent. These findings underscore the significance of serial position effects during the inference process, particularly in scenarios where there are no ground truth labels, highlighting the need for greater focus on addressing these effects in LLM applications.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 14 of its 26 references that are in the map. First posted on arXiv 23 Jun 2024.

Builds on

14 of 26 references.

References · 14

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  5. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  6. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  7. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  8. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  9. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  10. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  11. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  12. 2021Re-TACRED: Addressing Shortcomings of the TACRED Datasetcited by 74
2 more
  1. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  2. 2023Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approachcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 19; 2026: 10.

30 papers.

Sort:

By area:

  1. 2025On the Emergence of Position Bias in Transformerscited by 59
  2. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 26
  3. 2025How Many Instructions Can LLMs Follow at Once?cited by 20
  4. 2025Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMscited by 9
  5. 2025Playing games with Large language models: Randomness and strategycited by 7
  6. 2025Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Modelscited by 6
  7. 2024CURATe: Benchmarking Personalised Alignment of Conversational AI Assistantscited by 5
  8. 2025Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMscited by 5
  9. 2025Fragile Preferences: A Deep Dive Into Order Effects in Large Language Modelscited by 5
  10. 2026A Structural Theory of Position Bias in Transformerscited by 5
  11. 2025Emergence of the Primacy Effect in Structured State-Space Modelscited by 4
  12. 2025Attention Debiasing for Token Pruning in Vision Language Modelscited by 4
18 more
  1. 2025Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAGcited by 3
  2. 2025Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agentscited by 3
  3. 2025Beyond Sliding Windows: Learning to Manage Memory in Non-Markovian Environmentscited by 3
  4. 2025AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applicationscited by 3
  5. 2025On the Reasoning Capacity of AI Models and How to Quantify Itcited by 2
  6. 2025Analysis of LLM as a grammatical feature tagger for African American Englishcited by 2
  7. 2026How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalizationcited by 2
  8. 2025On Psychology of AI -- Does Primacy Effect Affect ChatGPT and Other LLMs?cited by 1
  9. 2026To Copy or Not to Copy: Copying Is Easier to Induce Than Recallcited by 1
  10. 2026PLR: Plackett-Luce for Reordering In-Context Learning Examplescited by 1
  11. 2026Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigationcited by 1
  12. 2025Token Homogenization under Positional Biascited by 0
  13. 2025Approximating Human Preferences Using a Multi-Judge Learned Systemcited by 0
  14. 2026Interpretable-by-Design Transformers via Architectural Stream Independencecited by 0
  15. 2026SOCIA-EVO: Automated Simulator Construction via Dual-Anchored Bi-Level Optimizationcited by 0
  16. 2026@skills: Attention is all you havecited by 0
  17. 2026The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systemscited by 0
  18. 2026SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphscited by 0