# Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs

arXiv: https://arxiv.org/abs/2606.21359 · HTML: https://mlsanity.com/paper/2606.21359/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 76 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 30 of 38)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023)
- [VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation](https://mlsanity.com/paper/2406.19276/) (2024)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [SciFact-Open: Towards open-domain scientific claim verification](https://mlsanity.com/paper/2210.13777/) (2022)
- [SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature](https://mlsanity.com/paper/2406.07835/) (2024)
- [HALoGEN: Fantastic LLM Hallucinations and Where to Find Them](https://mlsanity.com/paper/2501.08292/) (2025)
- [Understanding Finetuning for Factual Knowledge Extraction](https://mlsanity.com/paper/2406.14785/) (2024)
- [Better than Average: Paired Evaluation of NLP Systems](https://mlsanity.com/paper/2110.10746/) (2021)
- [Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications](https://mlsanity.com/paper/2109.14776/) (2021)
- [WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries](https://mlsanity.com/paper/2407.17468/) (2024)
- [The Law of Knowledge Overshadowing: Towards Understanding, Predicting, and Preventing LLM Hallucination](https://mlsanity.com/paper/2502.16143/) (2025)
- [HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL Conferences](https://mlsanity.com/paper/2601.18724/) (2026)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [An Audit on the Perspectives and Challenges of Hallucinations in NLP](https://mlsanity.com/paper/2404.07461/) (2024)
- [OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs](https://mlsanity.com/paper/2405.05583/) (2024)
- [GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checking](https://mlsanity.com/paper/2502.16514/) (2025)
- [VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts](https://mlsanity.com/paper/2505.09701/) (2025)
- [Comparing Knowledge Sources for Open-Domain Scientific Claim Verification](https://mlsanity.com/paper/2402.02844/) (2024)
- [Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics](https://mlsanity.com/paper/2501.14883/) (2025)
- [How Does Response Length Affect Long-Form Factuality](https://mlsanity.com/paper/2505.23295/) (2025)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses significant risks in this high stakes use-case. Prior scientific hallucination evaluation work remains largely restricted to the biomedical domain, treats hallucination as a binary task, and has not examined the growing family of scientifically fine-tuned LLMs. We address these gaps with SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, paired with a modular evaluation framework targeting three factuality hallucination types: unverifiability, overclaim, and attribution. Using a controlled minimal-pairing design, we evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base. Our results indicate that 1. Scientifically fine-tuned models exhibit degraded factual reliability across all hallucination types and scientific domains, and 2. Fine-tuned models are internally less confident yet linguistically more assertive. A human pilot study further reveals that current fact-checking tools show only modest agreement with expert judgments on scientific content, and that defining scientifically check-worthy claims remains contested even among human annotators. Our findings fundamentally challenge current methods of domain-specific fine-tuning for factuality and call for developing improved verification infrastructure for scientific content.