# How should the advent of large language models affect the practice of science?

arXiv: https://arxiv.org/abs/2312.03759 · HTML: https://mlsanity.com/paper/2312.03759/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Time Travel in LLMs: Tracing Data Contamination in Large Language Models](https://mlsanity.com/paper/2308.08493/) (2023)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level](https://mlsanity.com/paper/2112.15594/) (2021)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing](https://mlsanity.com/paper/2303.13367/) (2023)
- [AI-assisted coding: Experiments with GPT-4](https://mlsanity.com/paper/2304.13187/) (2023)
- [GPT-4 Can't Reason](https://mlsanity.com/paper/2308.03762/) (2023)
- [The Challenge of Value Alignment: from Fairer Algorithms to AI Safety](https://mlsanity.com/paper/2101.06060/) (2021)
- [Friend or Foe? Exploring the Implications of Large Language Models on the Science System](https://mlsanity.com/paper/2306.09928/) (2023)
- [Response: Emergent analogical reasoning in large language models](https://mlsanity.com/paper/2308.16118/) (2023)

## Built on by (most cited first, 8 of 8)

- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [In-context learning agents are asymmetric belief updaters](https://mlsanity.com/paper/2402.03969/) (2024)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas](https://mlsanity.com/paper/2503.19598/) (2025)
- [A Survey on Human-AI Collaboration with Large Foundation Models](https://mlsanity.com/paper/2403.04931/) (2024)
- [Information Retrieval in the Age of Generative AI: The RGB Model](https://mlsanity.com/paper/2504.20610/) (2025)
- [Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams](https://mlsanity.com/paper/2605.27404/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are being increasingly incorporated into scientific workflows. However, we have yet to fully grasp the implications of this integration. How should the advent of large language models affect the practice of science? For this opinion piece, we have invited four diverse groups of scientists to reflect on this query, sharing their perspectives and engaging in debate. Schulz et al. make the argument that working with LLMs is not fundamentally different from working with human collaborators, while Bender et al. argue that LLMs are often misused and over-hyped, and that their limitations warrant a focus on more specialized, easily interpretable tools. Marelli et al. emphasize the importance of transparent attribution and responsible use of LLMs. Finally, Botvinick and Gershman advocate that humans should retain responsibility for determining the scientific roadmap. To facilitate the discussion, the four perspectives are complemented with a response from each group. By putting these different perspectives in conversation, we aim to bring attention to important considerations within the academic community regarding the adoption of LLMs and their impact on both current and future scientific practices.