# Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2021

arXiv: https://arxiv.org/abs/2207.05133 · HTML: https://mlsanity.com/paper/2207.05133/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 59 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [RumourEval 2019: Determining Rumour Veracity and Support for Rumours](https://mlsanity.com/paper/1809.06683/) (2018)
- [BanFakeNews: A Dataset for Detecting Fake News in Bangla](https://mlsanity.com/paper/2004.08789/) (2020)
- [Stop Clickbait: Detecting and Preventing Clickbaits in Online News Media](https://mlsanity.com/paper/1610.09786/) (2016)
- [A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities](https://mlsanity.com/paper/1812.00315/) (2018)
- [An Emotional Analysis of False Information in Social Media and News Articles](https://mlsanity.com/paper/1908.09951/) (2019)
- [Neural Check-Worthiness Ranking with Weak Supervision: Finding Sentences for Fact-Checking](https://mlsanity.com/paper/1903.08404/) (2019)
- [Investigating Italian disinformation spreading on Twitter in the context of 2019 European elections](https://mlsanity.com/paper/1907.08170/) (2019)

## Built on by (most cited first, 4 of 4)

- [Monolingual and Multilingual Misinformation Detection for Low-Resource Languages: A Comprehensive Survey](https://mlsanity.com/paper/2410.18390/) (2024)
- [UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking](https://mlsanity.com/paper/2505.15063/) (2025)
- [Ax-to-Grind Urdu: Benchmark Dataset for Urdu Fake News Detection](https://mlsanity.com/paper/2403.14037/) (2024)
- [When Benchmarks Mislead: Shortcut Learning, Length Confounds, and the Limits of Cross-Dataset Generalization in Multilingual Fake News and Sarcasm Detection](https://mlsanity.com/paper/2607.14131/) (2026)

## Abstract (arXiv)

Automatic detection of fake news is a highly important task in the contemporary world. This study reports the 2nd shared task called UrduFake@FIRE2021 on identifying fake news detection in Urdu. The goal of the shared task is to motivate the community to come up with efficient methods for solving this vital problem, particularly for the Urdu language. The task is posed as a binary classification problem to label a given news article as a real or a fake news article. The organizers provide a dataset comprising news in five domains: (i) Health, (ii) Sports, (iii) Showbiz, (iv) Technology, and (v) Business, split into training and testing sets. The training set contains 1300 annotated news articles -- 750 real news, 550 fake news, while the testing set contains 300 news articles -- 200 real, 100 fake news. 34 teams from 7 different countries (China, Egypt, Israel, India, Mexico, Pakistan, and UAE) registered to participate in the UrduFake@FIRE2021 shared task. Out of those, 18 teams submitted their experimental results, and 11 of those submitted their technical reports, which is substantially higher compared to the UrduFake shared task in 2020 when only 6 teams submitted their technical reports. The technical reports submitted by the participants demonstrated different data representation techniques ranging from count-based BoW features to word vector embeddings as well as the use of numerous machine learning algorithms ranging from traditional SVM to various neural network architectures including Transformers such as BERT and RoBERTa. In this year's competition, the best performing system obtained an F1-macro score of 0.679, which is lower than the past year's best result of 0.907 F1-macro. Admittedly, while training sets from the past and the current years overlap to a large extent, the testing set provided this year is completely different.