Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit
Not on the map. Why?
Abstract (arXiv)
As large language models (LLMs) become default tools for online information verification, an implicit assumption follows them: that scale and general capability are sufficient for nuanced classification of misinformation discourse. We test this assumption directly on 900 Reddit comments spanning three PolitiFact-verified misinformation claims (environment, health, immigration), labelled as belief (propagates the claim), fact-check (corrects it), or other. We compare nine models across three paradigms -- BART-MNLI, three Llama variants, three commercial frontier LLMs (Claude Haiku 4.5, Gemini Flash Lite 2.5, Claude Sonnet 4.6), and fine-tuned DistilBERT and RoBERTa -- under universal and topic-specific label schemas. The assumption does not hold. Fine-tuned RoBERTa reaches 0.62 macro-F1 against a best zero-shot result of 0.50 (Claude Haiku 4.5), at a fraction of the per-query cost; the supervised advantage is concentrated on the belief class, the implicit, affective category every zero-shot model under-detects. Scaling does not help: Llama-3-8B matches Llama-3-70B, and Claude Sonnet 4.6 underperforms the smaller Haiku under generic labels, collapsing belief detection to 0.17 and refusing outright on a subset of comments flagged as sensitive. This is a safety-alignment artefact, not a capacity limit. Label schema and topic jointly shape zero-shot performance, with the same model varying by more than 0.13 macro-F1 across topics under matched labels. In a verification context, where missing belief is the costlier error, task-specific fine-tuning remains the more reliable choice despite the proliferation of large generative models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
24 of 41 references.
References · 24
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
- 2021A Survey on Automated Fact-Checkingcited by 376
- 2017Fake News Detection on Social Media: A Data Mining Perspectivecited by 334
12 more
- 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
- 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
- 2016Stance Detection with Bidirectional Conditional Encodingcited by 93
- 2017SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumourscited by 77
- 2021A Survey on Stance Detection for Mis- and Disinformation Identificationcited by 75
- 2018RumourEval 2019: Determining Rumour Veracity and Support for Rumourscited by 74
- 2015Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threadscited by 62
- 2016Stance and Sentiment in Tweetscited by 62
- 2018CASCADE: Contextual Sarcasm Detection in Online Discussion Forumscited by 26
- 2017Turing at SemEval-2017 Task 8: Sequential Approach to Rumour Stance Classification with Branch-LSTMcited by 20
- 2021Political Bias and Factualness in News Sharing across more than 100,000 Online Communitiescited by 9
- 2021Stance Detection with BERT Embeddings for Credibility Analysis of Information on Social Mediacited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.