How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Study
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
Abstract (arXiv)
In the early stages of scientific research, researchers rely on core scholarly judgments to identify relevant literature, assess credible evidence, and determine which directions merit pursuit. As AI tools become increasingly integrated into these early-stage workflows, the scholarly judgments that were once transparent and attributable to individual researchers become obscured, raising critical Responsible AI (RAI) concerns around accountability, transparency, and trust. Yet how these three dimensions manifest in real-time, in-situ scholarly practice remains largely unexplored. To address this gap, we conducted a think-aloud study with 15 researchers to examine how they used AI tools powered by large language models (LLMs) across early-stage research tasks, including literature exploration, synthesis, and research ideation. Our key findings address the tripartite constructs of accountability, transparency, and trust. First, the confident tone of AI outputs misrepresents epistemic uncertainty, making it more difficult for researchers (who are ultimately accountable) to identify which outputs require the greatest scrutiny. Second, opaque retrieval and content construction make provenance difficult to establish for transparency. Third, trust in AI is fragile, context-dependent, and easily eroded. In response, participant researchers were seen to develop compensatory strategies to restore scholarly judgment under uncertainty. Overall, our findings serve to contextualize AI-mediated research as a RAI problem grounded in lived researcher experience and motivate attention to deliberate AI integration that preserves accountability, supports transparency, and fosters informed trust.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 75 references that are in the map. First posted on arXiv 25 Apr 2026.
Builds on
11 of 75 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 112
- 2024LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptionscited by 45
- 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
- 2020Principles to Practices for Responsible AI: Closing the Gapcited by 25
- 2023Scientists' Perspectives on the Potential for Generative AI in their Fieldscited by 22
- 2024Large Language Models in Qualitative Research: Uses, Tensions, and Intentionscited by 19
- 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
- 2024OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Modelscited by 7
- 2025Generative Interfaces for Language Modelscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Checkpoints Are Not Enough: Trust Calibration in CoSLR, a Human-AI System for Systematic Literature Reviews
- 2026Diagnostic Foundation for Evaluating LLMs' Research Integrity as Co-Scientists
- 2026Thinking Through Signs: PEEL as a Semiotic Scaffolding for Epistemically Accountable AI-Enabled Research
- 2026"Nobody Did This": Contribution, Originality, and Accountability in Agent-Mediated Collaboration
- 2026Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
- 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing
- 2026Generative AI Expands the Intellectual Reach of Course Based Undergraduate Research Experiences (CUREs)