# Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance

arXiv: https://arxiv.org/abs/2501.11849 · HTML: https://mlsanity.com/paper/2501.11849/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 10 of its 45 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 10 of 10)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Fake News Detection on Social Media using Geometric Deep Learning](https://mlsanity.com/paper/1902.06673/) (2019)
- [Let Your Graph Do the Talking: Encoding Structured Data for LLMs](https://mlsanity.com/paper/2402.05862/) (2024)
- [Graph Neural Networks with Continual Learning for Fake News Detection from Social Media](https://mlsanity.com/paper/2007.03316/) (2020)
- [Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text](https://mlsanity.com/paper/2402.13415/) (2024)
- [Blessing or curse? A survey on the Impact of Generative AI on Fake News](https://mlsanity.com/paper/2404.03021/) (2024)
- [Exposing Influence Campaigns in the Age of LLMs: A Behavioral-Based AI Approach to Detecting State-Sponsored Trolls](https://mlsanity.com/paper/2210.08786/) (2022)
- [Use of Graph Neural Networks in Aiding Defensive Cyber Operations](https://mlsanity.com/paper/2401.05680/) (2024)
- [The Susceptibility Paradox in Online Social Influence](https://mlsanity.com/paper/2406.11553/) (2024)

## Built on by (most cited first, 4 of 4)

- [Unmasking Conversational Bias in AI Multiagent Systems](https://mlsanity.com/paper/2501.14844/) (2025)
- [HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL](https://mlsanity.com/paper/2503.16528/) (2025)
- [VeriMoA: A Mixture-of-Agents Framework for Spec-to-HDL Generation](https://mlsanity.com/paper/2510.27617/) (2025)
- [Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Survey](https://mlsanity.com/paper/2606.00136/) (2026)

## Abstract (arXiv)

Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the textual similarities of their posts. Despite their effectiveness in recognizing astroturf campaigns, these methods face significant challenges, notably the class imbalance in available training datasets. To mitigate this issue, recent methods usually resort to data augmentation or increasing the number of positive samples, which may not always be feasible or sufficient in real-world settings. Following a different path, in this paper, we propose a novel framework for identifying astroturf campaigns based solely on large language models (LLMs), introducing a Balanced Retrieval-Augmented Generation (Balanced RAG) component. Our approach first gives both textual information concerning the posts (in our case tweets) and the user interactions of the social network as input to a language model. Then, through prompt engineering and the proposed Balanced RAG method, it effectively detects coordinated disinformation campaigns on X (Twitter). The proposed framework does not require any training or fine-tuning of the language model. Instead, by strategically harnessing the strengths of prompt engineering and Balanced RAG, it facilitates LLMs to overcome the effects of class imbalance and effectively identify coordinated political campaigns. The experimental results demonstrate that by incorporating the proposed prompt engineering and Balanced RAG methods, our framework outperforms the traditional graph-based baselines, achieving 2x-3x improvements in terms of precision, recall and F1 scores.