# Lifelong Evolution: Collaborative Learning between Large and Small Language Models for Continuous Emergent Fake News Detection

arXiv: https://arxiv.org/abs/2506.04739 · HTML: https://mlsanity.com/paper/2506.04739/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks](https://mlsanity.com/paper/2011.01844/) (2020)
- [Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection](https://mlsanity.com/paper/2309.12247/) (2023)
- [Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks](https://mlsanity.com/paper/2310.10830/) (2023)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Embracing Domain Differences in Fake News: Cross-domain Fake News Detection using Multi-modal Data](https://mlsanity.com/paper/2102.06314/) (2021)
- [Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom](https://mlsanity.com/paper/2405.03371/) (2024)
- [DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection](https://mlsanity.com/paper/2402.10426/) (2024)
- [Unveiling the Generalization Power of Fine-Tuned Large Language Models](https://mlsanity.com/paper/2403.09162/) (2024)
- [A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm](https://mlsanity.com/paper/2310.12244/) (2023)
- [Memory-Guided Multi-View Multi-Domain Fake News Detection](https://mlsanity.com/paper/2206.12808/) (2022)
- [Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection](https://mlsanity.com/paper/2306.14728/) (2023)
- [Contrastive Domain Adaptation for Early Misinformation Detection: A Case Study on COVID-19](https://mlsanity.com/paper/2208.09578/) (2022)
- [Robust Domain Misinformation Detection via Multi-modal Feature Alignment](https://mlsanity.com/paper/2311.14315/) (2023)
- [Incremental Task Learning with Incremental Rank Updates](https://mlsanity.com/paper/2207.09074/) (2022)
- [GLEAN: Active Generalized Category Discovery with Diverse LLM Feedback](https://mlsanity.com/paper/2502.18414/) (2025)
- [Collaborative Evolution: Multi-Round Learning Between Large and Small Language Models for Emergent Fake News Detection](https://mlsanity.com/paper/2503.21127/) (2025)

## Abstract (arXiv)

The widespread dissemination of fake news on social media has significantly impacted society, resulting in serious consequences. Conventional deep learning methodologies employing small language models (SLMs) suffer from extensive supervised training requirements and difficulties adapting to evolving news environments due to data scarcity and distribution shifts. Large language models (LLMs), despite robust zero-shot capabilities, fall short in accurately detecting fake news owing to outdated knowledge and the absence of suitable demonstrations. In this paper, we propose a novel Continuous Collaborative Emergent Fake News Detection (C2EFND) framework to address these challenges. The C2EFND framework strategically leverages both LLMs' generalization power and SLMs' classification expertise via a multi-round collaborative learning framework. We further introduce a lifelong knowledge editing module based on a Mixture-of-Experts architecture to incrementally update LLMs and a replay-based continue learning method to ensure SLMs retain prior knowledge without retraining entirely. Extensive experiments on Pheme and Twitter16 datasets demonstrate that C2EFND significantly outperforms existed methods, effectively improving detection accuracy and adaptability in continuous emergent fake news scenarios.