Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing
Not on the map. Why?
Abstract (arXiv)
Large Language Models have seen expanding application across domains, yet their effectiveness as assistive tools for scientific writing - an endeavor requiring precision, multimodal synthesis, and domain expertise - remains insufficiently understood. We examine the potential of LLMs to support domain experts in scientific writing, with a focus on abstract composition. We design an incentivized randomized controlled trial with a hypothetical conference setup where participants with relevant expertise are split into an author and reviewer pool. Inspired by methods in behavioral science, our novel incentive structure encourages authors to edit the provided abstracts to an acceptable quality for a peer-reviewed submission. Our 2 x 2 between-subject design expands into two dimensions: the implicit source of the provided abstract and the disclosure of it. We find authors make most edits when editing human-written abstracts compared to AI-generated abstracts without source attribution, often guided by higher perceived readability in AI generation. Upon disclosure of source information, the volume of edits converges in both source treatments. Reviewer decisions remain unaffected by the source of the abstract, but bear a significant correlation with the number of edits made. Careful stylistic edits, especially in the case of AI-generated abstracts, in the presence of source information, improve the chance of acceptance. We find that AI-generated abstracts hold potential to reach comparable levels of acceptability to human-written ones with minimal revision, and that perceptions of AI authorship, rather than objective quality, drive much of the observed editing behavior. Our findings reverberate the significance of source disclosure in collaborative scientific writing.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 117 references that are in the map. First posted on arXiv 16 Nov 2025.
Builds on
46 of 117 references.
References · 46
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
- 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
- 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 124
- 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
- 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
- 2022Human heuristics for AI-generated language are flawedcited by 105
- 2024Are Large Language Models Capable of Generating Human-Level Narratives?cited by 79
- 2024Delving into LLM-assisted writing in biomedical publications through excess vocabularycited by 78
- 2024Hypothesis Generation with Large Language Modelscited by 71
- 2022Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writerscited by 69
34 more
- 2024Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Modelscited by 51
- 2019PaperRobot: Incremental Draft Generation of Scientific Ideascited by 37
- 2024Data-driven Discovery with Large Generative Modelscited by 34
- 2019How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?cited by 30
- 2024A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedianscited by 29
- 2023Accelerating science with human-aware artificial intelligencecited by 28
- 2024Learning to Plan and Generate Text with Citationscited by 28
- 2024Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?cited by 27
- 2025SciSciGPT: Advancing Human-AI Collaboration in the Science of Sciencecited by 27
- 2024Is ChatGPT Transforming Academics' Writing Style?cited by 26
- 2022CiteBench: A benchmark for Scientific Citation Text Generationcited by 22
- 2024StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planningcited by 20
- 2023"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Modelscited by 17
- 2024Related Work and Citation Text Generation: A Surveycited by 17
- 2019Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistancecited by 16
- 2024From Pen to Prompt: How Creative Writers Integrate AI into their Writing Practicecited by 14
- 2024"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Modelscited by 14
- 2025Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Processcited by 14
- 2024How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?cited by 12
- 2023A Gold Standard Dataset for the Reviewer Assignment Problemcited by 11
- 2024Scientific Hypothesis Generation by a Large Language Model: Laboratory Validation in Breast Cancer Treatmentcited by 11
- 2025Can Large Language Models Help Experimental Design for Causal Discovery?cited by 11
- 2025AI Safety Should Prioritize the Future of Workcited by 11
- 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 11
- 2023Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communicationcited by 9
- 2025How much are LLMs changing the language of academic papers after ChatGPT? A multi-database and full text analysiscited by 8
- 2025Multi-LLM Collaborative Caption Generation in Scientific Documentscited by 6
- 2024Locally Adaptive Neural 3D Morphable Modelscited by 5
- 2025A Systematic Review of Human-AI Co-Creativitycited by 4
- 2025Constella: Supporting Storywriters' Interconnected Character Creation through LLM-based Multi-Agentscited by 4
- 2025Patterns and Purposes: A Cross-Journal Analysis of AI Tool Usage in Academic Writingcited by 3
- 2025Towards AI-assisted Academic Writingcited by 3
- 2017Single versus Double Blind Reviewing at WSDM 2017cited by 2
- 2024A Perspective on Literary Metaphor in the Context of Generative AIcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.