# Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models

arXiv: https://arxiv.org/abs/2410.20940 · HTML: https://mlsanity.com/paper/2410.20940/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 59 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen](https://mlsanity.com/paper/2005.00701/) (2020)
- [BLESS: Benchmarking Large Language Models on Sentence Simplification](https://mlsanity.com/paper/2310.15773/) (2023)
- [DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking](https://mlsanity.com/paper/2004.12864/) (2020)
- [Fake News Detection via NLP is Vulnerable to Adversarial Attacks](https://mlsanity.com/paper/1901.09657/) (2019)
- [From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards](https://mlsanity.com/paper/2403.13213/) (2024)
- [Are Large Language Models Actually Good at Text Style Transfer?](https://mlsanity.com/paper/2406.05885/) (2024)
- [RAFT: Realistic Attacks to Fool Text Detectors](https://mlsanity.com/paper/2410.03658/) (2024)
- [Verifying the Robustness of Automatic Credibility Assessment](https://mlsanity.com/paper/2303.08032/) (2023)
- [The Daunting Task of Real-World Textual Style Transfer Auto-Evaluation](https://mlsanity.com/paper/1910.03747/) (2019)
- [How Vulnerable Are Automatic Fake News Detection Methods to Adversarial Attacks?](https://mlsanity.com/paper/2107.07970/) (2021)
- [OpenFact at CheckThat! 2024: Combining Multiple Attack Methods for Effective Adversarial Text Generation](https://mlsanity.com/paper/2409.02649/) (2024)
- [Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection](https://mlsanity.com/paper/1907.07033/) (2019)

## Built on by (most cited first, 10 of 10)

- [Adversarial Attacks Against Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2509.08463/) (2025)
- [CAMOUFLAGE: Exploiting Misinformation Detection Systems Through LLM-driven Adversarial Claim Transformation](https://mlsanity.com/paper/2505.01900/) (2025)
- [The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence](https://mlsanity.com/paper/2601.05478/) (2026)
- [Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection](https://mlsanity.com/paper/2501.14728/) (2025)
- [CogniDir: Combating Cognitive Malicious Comments via Adaptive Distributional Learning for Robust Fake News Detection](https://mlsanity.com/paper/2510.09712/) (2025)
- [Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach](https://mlsanity.com/paper/2512.17367/) (2025)
- [Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines](https://mlsanity.com/paper/2604.23483/) (2026)
- [Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Survey](https://mlsanity.com/paper/2606.00136/) (2026)
- [Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability](https://mlsanity.com/paper/2607.10402/) (2026)
- [Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks](https://mlsanity.com/paper/2608.04286/) (2026)

## Abstract (arXiv)

Large language models have many beneficial applications, but can they also be used to attack content-filtering algorithms in social media platforms? We investigate the challenge of generating adversarial examples to test the robustness of text classification algorithms detecting low-credibility content, including propaganda, false claims, rumours and hyperpartisan news. We focus on simulation of content moderation by setting realistic limits on the number of queries an attacker is allowed to attempt. Within our solution (TREPAT), initial rephrasings are generated by large language models with prompts inspired by meaning-preserving NLP tasks, such as text simplification and style transfer. Subsequently, these modifications are decomposed into small changes, applied through beam search procedure, until the victim classifier changes its decision. We perform (1) quantitative evaluation using various prompts, models and query limits, (2) targeted manual assessment of the generated text and (3) qualitative linguistic analysis. The results confirm the superiority of our approach in the constrained scenario, especially in case of long input text (news articles), where exhaustive search is not feasible.