mlsanityData to 14 Oct 2026

The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?

arXiv 2410.04699 ↗First posted 7 Oct 2024cs.CL cs.HCcited by 14

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown capabilities close to human performance in various analytical tasks, leading researchers to use them for time and labor-intensive analyses. However, their capability to handle highly specialized and open-ended tasks in domains like policy studies remains in question. This paper investigates the efficiency and accuracy of LLMs in specialized tasks through a structured user study focusing on Human-LLM partnership. The study, conducted in two stages-Topic Discovery and Topic Assignment-integrates LLMs with expert annotators to observe the impact of LLM suggestions on what is usually human-only analysis. Results indicate that LLM-generated topic lists have significant overlap with human generated topic lists, with minor hiccups in missing document-specific topics. However, LLM suggestions may significantly improve task completion speed, but at the same time introduce anchoring bias, potentially affecting the depth and nuance of the analysis, raising a critical question about the trade-off between increased efficiency and the risk of biased analysis.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 10 of its 33 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

10 of 33 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  5. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  6. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
  7. 2024Escalation Risks from Language Models in Military and Diplomatic Decision-Makingcited by 63
  8. 2023Global Voices, Local Biases: Socio-Cultural Prejudices across Languagescited by 23
  9. 2021Evaluating the Interpretability of Generative Models by Interactive Reconstructioncited by 17
  10. 2024Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025The Bias is in the Details: An Assessment of Cognitive Bias in LLMscited by 10
  2. 2025Quantifying Cognitive Bias Induction in LLM-Generated Contentcited by 7
  3. 2025Variance reduction in output from generative AIcited by 6
  4. 2025Political Ideology Shifts in Large Language Modelscited by 5
  5. 2025Unraveling Human-AI Teaming: A Review and Outlookcited by 4
  6. 2025Show or Tell? Modeling the evolution of request-making in Human-LLM conversationscited by 3
  7. 2025ROBoto2: An Interactive System and Dataset for LLM-assisted Clinical Trial Risk of Bias Assessmentcited by 3
  8. 2025Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Modelscited by 2
  9. 2025Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientationcited by 1
  10. 2026Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communitiescited by 1
  11. 2025Approximating the Mathematical Structure of Psychodynamicscited by 0
  12. 2026Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Makingcited by 0
2 more
  1. 2026WhoSaidIt: Human-LLM Collaborative Annotation for Text-Based Multilingual Speaker-Attribute Classificationcited by 0
  2. 2026Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writingcited by 0