# It's All in the Embedding! Fake News Detection Using Document Embeddings

arXiv: https://arxiv.org/abs/2304.07781 · HTML: https://mlsanity.com/paper/2304.07781/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 81 references that are in the map.
- First posted on arXiv 16 Apr 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [NELA-GT-2020: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2102.04567/) (2021)
- [A Systematic Media Frame Analysis of 1.5 Million New York Times Articles from 2000 to 2017](https://mlsanity.com/paper/2005.01803/) (2020)
- [Proppy: A System to Unmask Propaganda in Online News](https://mlsanity.com/paper/1912.06810/) (2019)
- [Meta-Path-based Fake News Detection Leveraging Multi-level Social Context Information](https://mlsanity.com/paper/2109.08022/) (2021)
- [Sentimental LIAR: Extended Corpus and Deep Learning Models for Fake Claim Classification](https://mlsanity.com/paper/2009.01047/) (2020)
- [The Fake News Spreading Plague: Was it Preventable?](https://mlsanity.com/paper/1703.06988/) (2017)
- [TextBenDS: a generic Textual data Benchmark for Distributed Systems](https://mlsanity.com/paper/2108.05689/) (2021)
- [Fake News Detection: a comparison between available Deep Learning techniques in vector space](https://mlsanity.com/paper/2102.09470/) (2021)

## Built on by (most cited first, 7 of 7)

- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs](https://mlsanity.com/paper/2505.02009/) (2025)
- [Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data](https://mlsanity.com/paper/2412.14276/) (2024)
- [Fake News Detection: It's All in the Data!](https://mlsanity.com/paper/2407.02122/) (2024)
- [StopHC: A Harmful Content Detection and Mitigation Architecture for Social Media Platforms](https://mlsanity.com/paper/2411.06138/) (2024)
- [DocNet: Semantic Structure in Inductive Bias Detection Models](https://mlsanity.com/paper/2406.10965/) (2024)
- [GETAE: Graph information Enhanced deep neural NeTwork ensemble ArchitecturE for fake news detection](https://mlsanity.com/paper/2412.01825/) (2024)

## Abstract (arXiv)

With the current shift in the mass media landscape from journalistic rigor to social media, personalized social media is becoming the new norm. Although the digitalization progress of the media brings many advantages, it also increases the risk of spreading disinformation, misinformation, and malformation through the use of fake news. The emergence of this harmful phenomenon has managed to polarize society and manipulate public opinion on particular topics, e.g., elections, vaccinations, etc. Such information propagated on social media can distort public perceptions and generate social unrest while lacking the rigor of traditional journalism. Natural Language Processing and Machine Learning techniques are essential for developing efficient tools that can detect fake news. Models that use the context of textual data are essential for resolving the fake news detection problem, as they manage to encode linguistic features within the vector representation of words. In this paper, we propose a new approach that uses document embeddings to build multiple models that accurately label news articles as reliable or fake. We also present a benchmark on different architectures that detect fake news using binary or multi-labeled classification. We evaluated the models on five large news corpora using accuracy, precision, and recall. We obtained better results than more complex state-of-the-art Deep Neural Network models. We observe that the most important factor for obtaining high accuracy is the document encoding, not the classification model's complexity.