mlsanityData to 7 Oct 2026

sciwrite-lint: Verification Infrastructure for the Age of Science Vibe-Writing

arXiv 2604.08501 ↗First posted 9 Apr 2026cs.DL cs.CL cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Scientific papers make claims about prior work backed by citations. Verifying those citations at scale (that each cited paper exists, says what the citation claims, and is itself reliable) is structurally beyond what human review can deliver: a typical paper has dozens of citations, and a careful reviewer reads at most a handful end-to-end. AI-assisted writing makes this gap even more urgent: LLMs hallucinate references and may fill in plausible details from titles or abstracts of papers they never read, worse for the smaller local-weights models that privacy-aware researchers must use. sciwrite-lint applies the linting paradigm from software engineering to citation verification: it runs entirely on the researcher's machine (free public databases, a single consumer GPU, and open-weights models), is fast enough to re-lint between revisions so authors catch problems at the source while drafting, and serves journals and reviewers as an automated first pass. The pipeline checks reference existence, metadata accuracy, retraction status, and claim support, traverses one level into cited papers' bibliographies, and produces per-reference reliability scores. We evaluate on 30 unseen papers (arXiv and bioRxiv) with error injection and LLM-adjudicated false-positive analysis. The same linting workflow extends to internal consistency: numbers in text vs. tables, abstract vs. body, figure captions vs. content, statistical results vs. their verbal interpretation, plus structural cross-references (dangling cites, orphan references). As a separate experimental contribution we also propose SciLint Score: citation-chain integrity combined with a contribution component operationalizing five philosophy-of-science frameworks (Popper, Lakatos, Kitcher, Laudan, Mayo).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 66 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

19 of 66 references.

References · 19

  1. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  3. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  4. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  5. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  6. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  7. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  8. 2023Inverse Scaling: When Bigger Isn't Bettercited by 205
  9. 2022Inverse scaling can become U-shapedcited by 76
  10. 2025Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?cited by 32
  11. 2026CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Eracited by 14
  12. 2026Compound Deception in Elite Peer Review: A Failure Mode Taxonomy of 100 Fabricated Citations at NeurIPS 2025cited by 13
7 more
  1. 2025A review on the novelty measurements of academic paperscited by 12
  2. 2019What Do Citation Counts Measure? An Updated Review of Studies on Citations in Scientific Documents Published between 2006 and 2018cited by 9
  3. 2026GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Modelscited by 9
  4. 2025SemanticCite: Citation Verification with AI-Powered Full-Text Analysis and Evidence-Based Reasoningcited by 3
  5. 2026The Case of the Mysterious Citationscited by 3
  6. 2026scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patternscited by 3
  7. 2013Deep Impact: Unintended consequences of journal rankcited by 1

Built on by

Not cited yet.