# Are LLMs More Skeptical of Entertainment News?

arXiv: https://arxiv.org/abs/2605.01727 · HTML: https://mlsanity.com/paper/2605.01727/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 3 May 2026.

## Builds on (most cited first, 25 of 25)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [A Stylometric Inquiry into Hyperpartisan and Fake News](https://mlsanity.com/paper/1702.05638/) (2017)
- [Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection](https://mlsanity.com/paper/2309.12247/) (2023)
- [Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks](https://mlsanity.com/paper/2310.10830/) (2023)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [The simulation of judgment in LLMs](https://mlsanity.com/paper/2502.04426/) (2025)
- [A Review of Text Style Transfer using Deep Learning](https://mlsanity.com/paper/2109.15144/) (2021)
- [Fake News Detectors are Biased against Texts Generated by Large Language Models](https://mlsanity.com/paper/2309.08674/) (2023)
- [Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals](https://mlsanity.com/paper/2309.07601/) (2023)
- [Adapting Fake News Detection to the Era of Large Language Models](https://mlsanity.com/paper/2311.04917/) (2023)
- [Explore the Potential of LLMs in Misinformation Detection: An Empirical Study](https://mlsanity.com/paper/2311.12699/) (2023)
- [Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation](https://mlsanity.com/paper/2401.06920/) (2024)
- [Can Large Language Models Detect Misinformation in Scientific News Reporting?](https://mlsanity.com/paper/2402.14268/) (2024)
- [Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection](https://mlsanity.com/paper/2506.02350/) (2025)
- [Robust Fake News Detection using Large Language Models under Adversarial Sentiment Attacks](https://mlsanity.com/paper/2601.15277/) (2026)
- [SoK: Machine Learning for Misinformation Detection](https://mlsanity.com/paper/2308.12215/) (2023)
- [SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer](https://mlsanity.com/paper/2406.04578/) (2024)
- [Variation between Credible and Non-Credible News Across Topics](https://mlsanity.com/paper/2411.12458/) (2024)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed standards across journalistic genres. We examine whether zero-shot LLMs are more likely to misclassify legitimate entertainment news as fake than legitimate hard news, using a within-dataset design on GossipCop from FakeNewsNet. Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both p < .001), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference. A style-swap experiment yields only limited and inconsistent changes, suggesting that the asymmetry is not reducible to stylistic register alone. Prompt-based mitigation is likewise possible but not generic: framing the model as an entertainment-news fact-checker reduces false positives for DeepSeek-V3.2 by about 50% without detectable recall loss, but offers little improvement for GPT-5.2. Exploratory qualitative coding further suggests two recurring error patterns in sampled false positives: treating private-life claims as inherently unverifiable and discounting entertainment journalism as an epistemically weaker genre. Taken together, these findings show that aggregate performance metrics can obscure structured false positives within legitimate journalism. We argue that LLM-based credibility assessment may not only evaluate truth claims but also differentially recognize the legitimacy of journalistic genres, and that evaluation should therefore include genre-stratified false-positive analysis alongside overall accuracy.