mlsanityData to 14 Oct 2026

Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain

arXiv 2609.07754 ↗First posted 7 Sep 2026cs.CR cs.AI cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

AI coding assistants now select, install, and configure software, and attackers have exploited that position through invented package names, compromised maintainer accounts, and manipulated repository text. In response, the supply-chain community publishes machine-checkable trust signals: software bills of materials, signed releases, build provenance attestations, and declared official channels. Whether coding assistants read or act on those signals has not been measured for any of these classes on research software. We pre-registered and ran a controlled study on six open-source research software projects (three HPC, three quantum computing) drawn from an 87-project corpus, with protocol, seed, panel, and analysis plan deposited with a DOI before any trial. W created nine modified copies for each project: no signal, one per signal class, two with a signature or attestation from the wrong issuer, one with all four signals, and one reproducing documented conflicts in the project's own metadata. Three models under two ways of operating an assistant, with and without an approval step, gave 1,920 registered trials, plus a supplement on three frontier models. We scored behavior from container logs rather than from what the assistant said, and recorded the cost of every trial. Verification was rare under every condition: in 9 of 1,920 registered trials (0.5%), the assistant opened any provenance signal before installing in 0 of 384 control trials, and no trial ran a verification command, so signal presence had no measurable effect. We drew three conclusions: publishing signals is necessary but not sufficient; price did not buy verification (the model that verified most often costs 0.10 per trial; the most capable, at 1.00, verified nothing); verification must be built into the program that runs the assistant. We release the per-trial cost ledger, the protocol, and every log.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 29 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

14 of 29 references.

References · 14

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 65
  3. 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
  4. 2026Auditing Provenance Sensitivity in LLM Agent Action Selectioncited by 12
  5. 2023An Empirical Study on Software Bill of Materials: Where We Stand and the Road Aheadcited by 7
  6. 2022OpenSSF Scorecard: On the Path Toward Ecosystem-wide Automated Security Metricscited by 5
  7. 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agentscited by 3
  8. 2026Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computingcited by 3
  9. 2024Analyzing Challenges in Deployment of the SLSA Framework for Software Supply Chain Securitycited by 2
  10. 2026Operationalizing Research Software for Supply Chain Securitycited by 2
  11. 2026Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flightcited by 2
  12. 2026Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versionscited by 1
2 more
  1. 2026Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agentscited by 1
  2. 2026A Multi-Surface Consistency Audit of Software Citation Metadatacited by 1

Built on by

Not cited yet.