# A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models arXiv: https://arxiv.org/abs/2410.21360 ยท HTML: https://mlsanity.com/paper/2410.21360/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 40 of its 110 references that are in the map. - First posted on arXiv 28 Oct 2024. ## Builds on (most cited first, 30 of 40) - [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022) - [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021) - [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017) - [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018) - [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022) - [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020) - [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020) - [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023) - [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023) - [We Can Detect Your Bias: Predicting the Political Ideology of News Articles](https://mlsanity.com/paper/2010.05338/) (2020) - [X-FACT: A New Benchmark Dataset for Multilingual Fact Checking](https://mlsanity.com/paper/2106.09248/) (2021) - [Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements](https://mlsanity.com/paper/2401.06766/) (2024) - [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020) - [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020) - [Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News](https://mlsanity.com/paper/2010.03159/) (2020) - [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2402.11175/) (2024) - [What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context](https://mlsanity.com/paper/2005.04518/) (2020) - [Claim Matching Beyond English to Scale Global Fact-Checking](https://mlsanity.com/paper/2106.00853/) (2021) - [Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective](https://mlsanity.com/paper/2210.10488/) (2022) - [Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research](https://mlsanity.com/paper/2401.11969/) (2024) - [Combating Fake News: A Survey on Identification and Mitigation Techniques](https://mlsanity.com/paper/1901.06437/) (2019) - [Analyzing Political Bias and Unfairness in News Articles at Different Levels of Granularity](https://mlsanity.com/paper/2010.10652/) (2020) - [Disinformation Capabilities of Large Language Models](https://mlsanity.com/paper/2311.08838/) (2023) - [Prta: A System to Support the Analysis of Propaganda Techniques in the News](https://mlsanity.com/paper/2005.05854/) (2020) - [LESA: Linguistic Encapsulation and Semantic Amalgamation Based Generalised Claim Detection from Online Content](https://mlsanity.com/paper/2101.11891/) (2021) - [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://mlsanity.com/paper/2211.10057/) (2022) - [Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation](https://mlsanity.com/paper/2401.06643/) (2024) - [ApplicaAI at SemEval-2020 Task 11: On RoBERTa-CRF, Span CLS and Whether Self-Training Helps Them](https://mlsanity.com/paper/2005.07934/) (2020) - [Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2112.10322/) (2021) - [A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomness](https://mlsanity.com/paper/2312.01082/) (2023) ## Built on by (most cited first, 3 of 3) - [ClaimPT: A Portuguese Dataset of Annotated Claims in News Articles](https://mlsanity.com/paper/2601.19490/) (2026) - [Evidence Triangulation for Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2606.31367/) (2026) - [From Articles to Publishers: Aggregating Language Model Predictions for News Source Reliability Inference](https://mlsanity.com/paper/2609.24219/) (2026) ## Abstract (arXiv) In the age of social media and generative AI, the ability to automatically assess the credibility of online content has become increasingly critical, complementing traditional approaches to false information detection. Credibility assessment relies on aggregating diverse credibility signals - small units of information, such as content subjectivity, bias, or a presence of persuasion techniques - into a final credibility label/score. However, current research in automatic credibility assessment and credibility signals detection remains highly fragmented, with many signals studied in isolation and lacking integration. Notably, there is a scarcity of approaches that detect and aggregate multiple credibility signals simultaneously. These challenges are further exacerbated by the absence of a comprehensive and up-to-date overview of research works that connects these research efforts under a common framework and identifies shared trends, challenges, and open problems. In this survey, we address this gap by presenting a systematic and comprehensive literature review of 175 research papers, focusing on textual credibility signals within the field of Natural Language Processing (NLP), which undergoes a rapid transformation due to advancements in Large Language Models (LLMs). While positioning the NLP research into the the broader multidisciplinary landscape, we examine both automatic credibility assessment methods as well as the detection of nine categories of credibility signals. We provide an in-depth analysis of three key categories: 1) factuality, subjectivity and bias, 2) persuasion techniques and logical fallacies, and 3) check-worthy and fact-checked claims. In addition to summarising existing methods, datasets, and tools, we outline future research direction and emerging opportunities, with particular attention to evolving challenges posed by generative AI.