mlsanityData to 13 Oct 2026

CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Science

arXiv 2608.28631 ↗First posted 5 Aug 2026cs.AI cs.CE cs.CYcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

An AI scientist should not grade its own homework. Yet in the systems we examined, the agent that reviews the work usually comes from the same model family as the agent that produced it, or at least from the same vendor. Model evaluators are known to favour their own generations. Whether models trained alike also share blind spots is a conjecture, not a settled finding, but if they do, the reviewer inherits the author's. The record of what was flagged and what was waved through often sits in platform logs that nobody outside can replay. We present CrossAudit, a protocol for supervising autonomous research pipelines. It rests on three commitments. Each increment of work is audited by an agent from a different vendor against a rulebook a human wrote and versioned. Reports, verdicts, disputes and rulings are git commits, so the supervision history can be re-read and cited; raw model exchanges are not yet part of that record. Scripted checks run before any model does. Advisory judgement never gates the pipeline: a model blocks only by citing a rule, and no model may waive a deterministic failure. Blockers that survive a bounded number of revision rounds go to a person. We state the protocol as eight invariants. We describe a reference implementation built from GitHub Actions and a few hundred lines of Python, and report a live deployment of a closely related variant in a computational-chemistry pipeline. We also ran a seeded-defect trial (30 increments, 43 seeded defects, one run per configuration). A cross-vendor audit of our own repository then voided its blinding. We adopt that audit's findings and report the corrected results. The trial shows that two vendors read the same rulebook differently. It does not show that either is better. The strongest evidence here is the committed, uncontrolled record of cross-vendor audits of this paper itself.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 22 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

17 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
  2. 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents22 Feb
  3. 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
  4. 2024Language agents achieve superhuman synthesis of scientific knowledge10 Sep

Other references · 13

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,492
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,011
  3. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,041
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 1,018
  5. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 664
  6. 2025Accelerating scientific discovery with Co-Scientistcited by 383
  7. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 313
  8. 2026Towards End-to-End Automation of AI Researchcited by 152
  9. 2025Kosmos: An AI Scientist for Autonomous Discoverycited by 87
  10. 2025Do LLM Evaluators Prefer Themselves for a Reason?cited by 38
  11. 2026Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Biascited by 25
  12. 2026Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluationscited by 5
1 more
  1. 2022The 'Problematic Paper Screener' automatically selects suspect publications for post-publication (re)assessmentcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors
  2. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
  3. 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agents
  4. 2026Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research
  5. 2026Auditing Discovery Claims: A Two-Sided Criterion for Agentic Science, with the Negative Side Decidable
  6. 2026Automated Benchmark Auditing for AI Agents and Large Language Models
  7. 2026When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI
  8. 2026OpenDiscoveryTrace: Process Traces for Evaluating AI Scientist Workflows