# Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-the-Loop Strategy arXiv: https://arxiv.org/abs/2404.10259 ยท HTML: https://mlsanity.com/paper/2404.10259/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 82 references that are in the map. - First posted on arXiv 16 Apr 2024. ## Builds on (most cited first, 30 of 30) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019) - [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022) - [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022) - [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020) - [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022) - [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020) - [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023) - [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023) - [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023) - [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021) - [LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis](https://mlsanity.com/paper/2310.15100/) (2023) - [The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants](https://mlsanity.com/paper/1708.01425/) (2017) - [Political Discourse on Social Media: Echo Chambers, Gatekeepers, and the Price of Bipartisanship](https://mlsanity.com/paper/1801.01665/) (2018) - [A Holistic Framework for Analyzing the COVID-19 Vaccine Debate](https://mlsanity.com/paper/2205.01817/) (2022) - [Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections](https://mlsanity.com/paper/2305.05094/) (2023) - [Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the model](https://mlsanity.com/paper/2305.13014/) (2023) - [Inside the Right-Leaning Echo Chambers: Characterizing Gab, an Unmoderated Social System](https://mlsanity.com/paper/1807.03688/) (2018) - [On Microtargeting Socially Divisive Ads: A Case Study of Russia-Linked Ad Campaigns on Facebook](https://mlsanity.com/paper/1808.09218/) (2018) - [Understanding COVID-19 Vaccine Campaign on Facebook using Minimal Supervision](https://mlsanity.com/paper/2210.10031/) (2022) - [Clandestino or Rifugiato? Anti-immigration Facebook Ad Targeting in Italy](https://mlsanity.com/paper/2103.09224/) (2021) - [Weakly Supervised Learning for Analyzing Political Campaigns on Facebook](https://mlsanity.com/paper/2210.10669/) (2022) - [Discovering Latent Themes in Social Media Messaging: A Machine-in-the-Loop Approach Integrating LLMs](https://mlsanity.com/paper/2403.10707/) (2024) - [Does Yoga Make You Happy? Analyzing Twitter User Happiness using Textual and Temporal Information](https://mlsanity.com/paper/2012.02939/) (2020) - [RuArg-2022: Argument Mining Evaluation](https://mlsanity.com/paper/2206.09249/) (2022) - [Analysis of Climate Campaigns on Social Media using Bayesian Model Averaging](https://mlsanity.com/paper/2305.06174/) (2023) - [Characterization of Local Attitudes Toward Immigration Using Social Media](https://mlsanity.com/paper/1903.05072/) (2019) ## Built on by (most cited first, 9 of 9) - [RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services](https://mlsanity.com/paper/2507.10605/) (2025) - [Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluation](https://mlsanity.com/paper/2410.05401/) (2024) - [Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text](https://mlsanity.com/paper/2601.17172/) (2026) - [Can LLMs Assist Annotators in Identifying Morality Frames? -- Case Study on Vaccination Debate on Social Media](https://mlsanity.com/paper/2502.01991/) (2025) - [Iterative Topic Taxonomy Induction with LLMs: A Case Study of Electoral Advertising](https://mlsanity.com/paper/2510.15125/) (2025) - [Utilising Large Language Models for Generating Effective Counter Arguments to Anti-Vaccine Tweets](https://mlsanity.com/paper/2510.16359/) (2025) - [Paid Voices vs. Public Feeds: Interpretable Cross-Platform Theme-Based Analysis of Climate Discourse](https://mlsanity.com/paper/2601.13317/) (2026) - [Investigating Vaccine Buyer's Remorse: Post-Vaccination Decision Regret in COVID-19 Social Media Using Politically Diverse Human Annotation](https://mlsanity.com/paper/2604.09626/) (2026) - [Count Evidence, Not Sentences: Tempered Evidence Fusion of LLM Judgments for Long-Text Value Measurement](https://mlsanity.com/paper/2609.27165/) (2026) ## Abstract (arXiv) The widespread use of social media has led to a surge in popularity for automated methods of analyzing public opinion. Supervised methods are adept at text categorization, yet the dynamic nature of social media discussions poses a continual challenge for these techniques due to the constant shifting of the focus. On the other hand, traditional unsupervised methods for extracting themes from public discourse, such as topic modeling, often reveal overarching patterns that might not capture specific nuances. Consequently, a significant portion of research into social media discourse still depends on labor-intensive manual coding techniques and a human-in-the-loop approach, which are both time-consuming and costly. In this work, we study the problem of discovering arguments associated with a specific theme. We propose a generic LLMs-in-the-Loop strategy that leverages the advanced capabilities of Large Language Models (LLMs) to extract latent arguments from social media messaging. To demonstrate our approach, we apply our framework to contentious topics. We use two publicly available datasets: (1) the climate campaigns dataset of 14k Facebook ads with 25 themes and (2) the COVID-19 vaccine campaigns dataset of 9k Facebook ads with 14 themes. Additionally, we design a downstream task as stance prediction by leveraging talking points in climate debates. Furthermore, we analyze demographic targeting and the adaptation of messaging based on real-world events.