# LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions

arXiv: https://arxiv.org/abs/2411.05025 · HTML: https://mlsanity.com/paper/2411.05025/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 101 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research](https://mlsanity.com/paper/2402.00159/) (2024)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [MARG: Multi-Agent Review Generation for Scientific Papers](https://mlsanity.com/paper/2401.04259/) (2024)
- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates](https://mlsanity.com/paper/2405.02150/) (2024)
- [On the Societal Impact of Open Foundation Models](https://mlsanity.com/paper/2403.07918/) (2024)
- [Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science](https://mlsanity.com/paper/2305.15041/) (2023)
- [CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support](https://mlsanity.com/paper/2407.16148/) (2024)
- [Scientists' Perspectives on the Potential for Generative AI in their Fields](https://mlsanity.com/paper/2304.01420/) (2023)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [Friend or Foe? Exploring the Implications of Large Language Models on the Science System](https://mlsanity.com/paper/2306.09928/) (2023)
- ["I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices](https://mlsanity.com/paper/2403.19876/) (2024)
- [Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Media](https://mlsanity.com/paper/2305.12265/) (2023)
- [Gender, Age, and Technology Education Influence the Adoption and Appropriation of LLMs](https://mlsanity.com/paper/2310.06556/) (2023)
- [The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces](https://mlsanity.com/paper/2303.14334/) (2023)
- [From human-centered to social-centered artificial intelligence: Assessing ChatGPT's impact through disruptive events](https://mlsanity.com/paper/2306.00227/) (2023)
- [Qlarify: Recursively Expandable Abstracts for Directed Information Retrieval over Scientific Papers](https://mlsanity.com/paper/2310.07581/) (2023)
- [Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Research](https://mlsanity.com/paper/2303.07304/) (2023)

## Built on by (most cited first, 30 of 47)

- [PaSa: An LLM Agent for Comprehensive Academic Paper Search](https://mlsanity.com/paper/2501.10120/) (2025)
- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios](https://mlsanity.com/paper/2508.17692/) (2025)
- [Human-LLM Coevolution: Evidence from Academic Writing](https://mlsanity.com/paper/2502.09606/) (2025)
- [Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users](https://mlsanity.com/paper/2503.22610/) (2025)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [The Evolution of LLM Adoption in Industry Data Curation Practices](https://mlsanity.com/paper/2412.16089/) (2024)
- [Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing](https://mlsanity.com/paper/2507.01418/) (2025)
- [When Incentives Backfire, Data Stops Being Human](https://mlsanity.com/paper/2502.07732/) (2025)
- [GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses](https://mlsanity.com/paper/2604.11924/) (2026)
- [Research Borderlands: Analysing Writing Across Research Cultures](https://mlsanity.com/paper/2506.00784/) (2025)
- [Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Models](https://mlsanity.com/paper/2601.05144/) (2026)
- [LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv](https://mlsanity.com/paper/2601.17036/) (2026)
- [Do LLMs Favor LLMs? Quantifying Interaction Effects in Peer Review](https://mlsanity.com/paper/2601.20920/) (2026)
- [Large language models struggle with ethnographic text annotation](https://mlsanity.com/paper/2601.12099/) (2026)
- [Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation](https://mlsanity.com/paper/2602.08873/) (2026)
- [PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A](https://mlsanity.com/paper/2602.21045/) (2026)
- [Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users](https://mlsanity.com/paper/2603.16120/) (2026)
- [Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraud](https://mlsanity.com/paper/2607.10712/) (2026)
- [Most biomedical publications show signs of LLM-assisted writing](https://mlsanity.com/paper/2608.10715/) (2026)
- [Authorship Without Writing: Large Language Models and the Senior Author Analogy](https://mlsanity.com/paper/2509.05390/) (2025)
- [Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation](https://mlsanity.com/paper/2511.04706/) (2025)
- [From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews](https://mlsanity.com/paper/2512.11661/) (2025)
- [What Are We Measuring in NLG? A Meta-Analysis of Evaluation Trends 2020-2025](https://mlsanity.com/paper/2601.07648/) (2026)
- [Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025](https://mlsanity.com/paper/2603.09654/) (2026)
- [The Shrinking Lifespan of LLMs in Science](https://mlsanity.com/paper/2604.07530/) (2026)
- [Accelerating Antibiotic Discovery with Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2503.16655/) (2025)
- [Deconstructing Jazz Piano Style Using Machine Learning](https://mlsanity.com/paper/2504.05009/) (2025)
- [A computational model and tool for generating more novel opportunities in professional innovation processes](https://mlsanity.com/paper/2510.20402/) (2025)
- [Can GPT replace human raters? Validity and reliability of machine-generated norms for metaphors](https://mlsanity.com/paper/2512.12444/) (2025)

## Abstract (arXiv)

The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipeline, while others have urged caution due to risks and ethical concerns. Yet little work has sought to quantify and characterize how researchers use LLMs and why. We present the first large-scale survey of 816 verified research article authors to understand how the research community leverages and perceives LLMs as research tools. We examine participants' self-reported LLM usage, finding that 81% of researchers have already incorporated LLMs into different aspects of their research workflow. We also find that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity. However, women, non-binary, and senior researchers have greater ethical concerns, potentially hindering adoption.