Does a Tool Result Carry More Authority Than Plain Text? Three Prospective Studies of False-Claim Adoption in a Synthetic Assignment Task with Claude Opus 5
Not on the map. Why?
Abstract (arXiv)
Language-model systems increasingly read from stores they also write to, so a claim that was merely written earlier can return looking retrieved. We tested whether the message package carrying an unsupported assignment changes which answer a model gives in a synthetic lookup task. Claude Opus 5 selected a color code for a named item or abstained. In an exploratory four-arm study, false-code adoption was 0/24 with no target claim, 0/22 scorable trials when a prior assistant assertion named the target, 14/24 when a tool-result record named it, and 15/24 when that result used a ten-field metadata wrapper that marked it unchecked. The tool-result arm selected the record's code in 11/12 supported trials and 14/24 unsupported trials, ruling out a fixed output-token bias while leaving substantial planted-token heterogeneity. A document-preregistered replication reproduced the tool-result versus assistant-assertion gap, 7/24 against 0/24, one-sided Fisher exact p = 0.0047. The tool-result rate nevertheless fell from 14/24 to 7/24 across runs made four days apart. A second preregistered study gave the earlier comparison a live text control: both records were announced in advance and placed in the same final user turn, then target binding was swapped between the linked tool result and later inline JSON. Inline text was sufficient for false-code adoption in 60/60 trials; the tool-result condition produced 57/60, so the registered result-first superiority criterion failed, p = 1. The result does not show that tool results have no effect. It shows that native tool-result placement was not necessary and that this experiment did not find greater behavioral weight for the result package than for announced inline text. The findings concern a single model on one synthetic task template, accessed through one API.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 15 Aug 2026.
Builds on
23 of 33 references.
References · 23
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
11 more
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
- 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
- 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
- 2023Context-faithful Prompting for Large Language Modelscited by 76
- 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
- 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
- 2021Preregistering NLP Researchcited by 24
- 2026Who Do LLMs Trust? Human Experts Matter More Than Other LLMscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.