mlsanityData to 7 Oct 2026

LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions

arXiv 2411.05025 ↗First posted 30 Oct 2024cs.CL cs.AI cs.CYcited by 45

Not on the map. Why?

Abstract (arXiv)

The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipeline, while others have urged caution due to risks and ethical concerns. Yet little work has sought to quantify and characterize how researchers use LLMs and why. We present the first large-scale survey of 816 verified research article authors to understand how the research community leverages and perceives LLMs as research tools. We examine participants' self-reported LLM usage, finding that 81% of researchers have already incorporated LLMs into different aspects of their research workflow. We also find that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity. However, women, non-binary, and senior researchers have greater ethical concerns, potentially hindering adoption.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 21 of its 101 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

21 of 101 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Galactica: A Large Language Model for Sciencecited by 736
  3. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  4. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 112
  5. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  6. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 98
  7. 2024Delving into LLM-assisted writing in biomedical publications through excess vocabularycited by 75
  8. 2024The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Ratescited by 66
  9. 2024On the Societal Impact of Open Foundation Modelscited by 64
  10. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Sciencecited by 37
  11. 2024CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Supportcited by 28
  12. 2023Scientists' Perspectives on the Potential for Generative AI in their Fieldscited by 22
9 more
  1. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  2. 2023Friend or Foe? Exploring the Implications of Large Language Models on the Science Systemcited by 15
  3. 2024"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practicescited by 14
  4. 2023Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Mediacited by 11
  5. 2023The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfacescited by 9
  6. 2023Gender, Age, and Technology Education Influence the Adoption and Appropriation of LLMscited by 9
  7. 2023From human-centered to social-centered artificial intelligence: Assessing ChatGPT's impact through disruptive eventscited by 8
  8. 2023Qlarify: Recursively Expandable Abstracts for Directed Information Retrieval over Scientific Paperscited by 8
  9. 2023Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Researchcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 15; 2026: 28.

45 papers.

Sort:

By area:

  1. 2025PaSa: An LLM Agent for Comprehensive Academic Paper Searchcited by 61
  2. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
  3. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  4. 2025Human-LLM Coevolution: Evidence from Academic Writingcited by 15
  5. 2025Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Userscited by 11
  6. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  7. 2024The Evolution of LLM Adoption in Industry Data Curation Practicescited by 6
  8. 2025Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writingcited by 5
  9. 2025When Incentives Backfire, Data Stops Being Humancited by 4
  10. 2025Research Borderlands: Analysing Writing Across Research Culturescited by 3
  11. 2026Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Modelscited by 3
  12. 2026LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXivcited by 3
33 more
  1. 2026GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responsescited by 3
  2. 2026Do LLMs Favor LLMs? Quantifying Interaction Effects in Peer Reviewcited by 2
  3. 2026Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendationcited by 2
  4. 2026PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&Acited by 2
  5. 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
  6. 2026Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraudcited by 2
  7. 2026Most biomedical publications show signs of LLM-assisted writingcited by 2
  8. 2025Authorship Without Writing: Large Language Models and the Senior Author Analogycited by 1
  9. 2025Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientationcited by 1
  10. 2025From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviewscited by 1
  11. 2026Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025cited by 1
  12. 2026The Shrinking Lifespan of LLMs in Sciencecited by 1
  13. 2025Accelerating Antibiotic Discovery with Large Language Models and Knowledge Graphscited by 0
  14. 2025Deconstructing Jazz Piano Style Using Machine Learningcited by 0
  15. 2025A computational model and tool for generating more novel opportunities in professional innovation processescited by 0
  16. 2025Can GPT replace human raters? Validity and reliability of machine-generated norms for metaphorscited by 0
  17. 2026Conversational AI for Rapid Scientific Prototyping: A Case Study on ESA's ELOPE Competitioncited by 0
  18. 2026What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025cited by 0
  19. 2026Large language models struggle with ethnographic text annotationcited by 0
  20. 2026PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficientcited by 0
  21. 2026Large Language Models for Large-Scale, Rigorous Qualitative Analysis in Applied Health Services Researchcited by 0
  22. 2026LLM or Human? Perceptions of Trust and Information Quality in Research Summariescited by 0
  23. 2026MARS: Margin and Semantic-Aware Data Augmentation for Reward Modelingcited by 0
  24. 2026SemBench: A Universal Semantic Framework for LLM Evaluationcited by 0
  25. 2026How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Studycited by 0
  26. 2026Explain Like I'm 5 or Whatever I Choose: Evaluating the Interactive Potential of Language Model Responsescited by 0
  27. 2026Thinking Like a Scientist? A Structural Study of LLM-Generated Research Methodscited by 0
  28. 2026Validating LLMs in social science: Epistemic threats and emerging normscited by 0
  29. 2026Scientific exploration, collaboration and labor division in the large language model eracited by 0
  30. 2026Expectations and Practices around AI Disclosure in CS Researchcited by 0
  31. 2026Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflowscited by 0
  32. 2026Adaptive Multi-Value Control in LLMs via Causal Activation Steeringcited by 0
  33. 2026Ream: Unfolding Mutual Awareness in Human-Agent Workspacescited by 0