# Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake News Detection

arXiv: https://arxiv.org/abs/2312.01006 · HTML: https://mlsanity.com/paper/2312.01006/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 39 of its 68 references that are in the map.
- First posted on arXiv 2 Dec 2023.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 39)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Zero-shot Knowledge Transfer via Adversarial Belief Matching](https://mlsanity.com/paper/1905.09768/) (2019)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [CSI: A Hybrid Deep Model for Fake News Detection](https://mlsanity.com/paper/1703.06959/) (2017)
- [FANG: Leveraging Social Context for Fake News Detection Using Graph Representation](https://mlsanity.com/paper/2008.07939/) (2020)
- [Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching](https://mlsanity.com/paper/2102.02973/) (2021)
- [User Preference-aware Fake News Detection](https://mlsanity.com/paper/2104.12259/) (2021)
- [Incorporating Priors with Feature Attribution on Text Classification](https://mlsanity.com/paper/1906.08286/) (2019)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [Reinforced Multi-Teacher Selection for Knowledge Distillation](https://mlsanity.com/paper/2012.06048/) (2020)
- [Fair Feature Distillation for Visual Recognition](https://mlsanity.com/paper/2106.04411/) (2021)
- [Embracing Domain Differences in Fake News: Cross-domain Fake News Detection using Multi-modal Data](https://mlsanity.com/paper/2102.06314/) (2021)
- [Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning](https://mlsanity.com/paper/2009.13028/) (2020)
- [FIND: Human-in-the-Loop Debugging Deep Text Classifiers](https://mlsanity.com/paper/2010.04987/) (2020)
- [Generalizing to the Future: Mitigating Entity Bias in Fake News Detection](https://mlsanity.com/paper/2204.09484/) (2022)
- [MM-COVID: A Multilingual and Multimodal Data Repository for Combating COVID-19 Disinformation](https://mlsanity.com/paper/2011.04088/) (2020)
- [Zoom Out and Observe: News Environment Perception for Fake News Detection](https://mlsanity.com/paper/2203.10885/) (2022)
- [Memory-Guided Multi-View Multi-Domain Fake News Detection](https://mlsanity.com/paper/2206.12808/) (2022)
- [FREDOM: Fairness Domain Adaptation Approach to Semantic Scene Understanding](https://mlsanity.com/paper/2304.02135/) (2023)
- [Multimodal Emergent Fake News Detection via Meta Neural Process Networks](https://mlsanity.com/paper/2106.13711/) (2021)
- [Can Model Compression Improve NLP Fairness](https://mlsanity.com/paper/2201.08542/) (2022)

## Built on by (most cited first, 2 of 2)

- [Biasing & Debiasing based Approach Towards Fair Knowledge Transfer for Equitable Skin Analysis](https://mlsanity.com/paper/2405.10256/) (2024)
- [Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting](https://mlsanity.com/paper/2512.19735/) (2025)

## Abstract (arXiv)

Multi-domain fake news detection aims to identify whether various news from different domains is real or fake and has become urgent and important. However, existing methods are dedicated to improving the overall performance of fake news detection, ignoring the fact that unbalanced data leads to disparate treatment for different domains, i.e., the domain bias problem. To solve this problem, we propose the Dual-Teacher De-biasing Distillation framework (DTDBD) to mitigate bias across different domains. Following the knowledge distillation methods, DTDBD adopts a teacher-student structure, where pre-trained large teachers instruct a student model. In particular, the DTDBD consists of an unbiased teacher and a clean teacher that jointly guide the student model in mitigating domain bias and maintaining performance. For the unbiased teacher, we introduce an adversarial de-biasing distillation loss to instruct the student model in learning unbiased domain knowledge. For the clean teacher, we design domain knowledge distillation loss, which effectively incentivizes the student model to focus on representing domain features while maintaining performance. Moreover, we present a momentum-based dynamic adjustment algorithm to trade off the effects of two teachers. Extensive experiments on Chinese and English datasets show that the proposed method substantially outperforms the state-of-the-art baseline methods in terms of bias metrics while guaranteeing competitive performance.