mlsanityData to 14 Oct 2026

Authority Bias in Conversational Search Engines for Academic Paper Recommendation

arXiv 2609.00248 ↗First posted 31 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly used as conversational search engines for academic literature, yet whether they judge papers on content or on authority signals has not been tested causally. We investigate authority bias: systematic preference for papers based on author prestige, venue, and citations rather than content. Holding title and abstract constant, we vary authority metadata across three counterfactual conditions (original, flipped, boosted) over eight LLMs (five open-weight and three frontier closed-weight) in an in-context, single-turn, top-1 recommendation setting. Our experiments show that authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt-level debiasing. We further document a say-do gap: debiasing instructions suppress authority mentions far faster than authority-driven flips, so surface auditing systematically underestimates behavioral bias.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 56 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

23 of 56 references.

References · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2024A Survey on LLM-as-a-Judgecited by 1,423
  8. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  9. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  10. 2023A Survey on Large Language Models for Recommendationcited by 379
  11. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  12. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
11 more
  1. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
  2. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 124
  3. 2018Construction of the Literature Graph in Semantic Scholarcited by 118
  4. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  5. 2020The Connection Between Popularity Bias, Calibration, and Fairness in Recommendationcited by 35
  6. 2024Large Language Models as Recommender Systems: A Study of Popularity Biascited by 26
  7. 2024Large Language Models Reflect Human Citation Patterns with a Heightened Citation Biascited by 17
  8. 2025Who Gets Cited? Gender- and Majority-Bias in LLM-Driven Reference Selectioncited by 6
  9. 2025Whose Name Comes Up? Auditing LLM-Based Scholar Recommendationscited by 5
  10. 2025Prestige over merit: An adapted audit of LLM bias in peer reviewcited by 5
  11. 2025MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoningcited by 3

Built on by

Not cited yet.