# Closing the Loop: Testing ChatGPT to Generate Model Explanations to Improve Human Labelling of Sponsored Content on Social Media

arXiv: https://arxiv.org/abs/2306.05115 · HTML: https://mlsanity.com/paper/2306.05115/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?](https://mlsanity.com/paper/1912.08320/) (2019)
- ["Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction](https://mlsanity.com/paper/2210.03735/) (2022)
- ["Garbage In, Garbage Out" Revisited: What Do Machine Learning Application Papers Report About Human-Labeled Training Data?](https://mlsanity.com/paper/2107.02278/) (2021)
- [An Empirical Study of Affiliate Marketing Disclosures on YouTube and Pinterest](https://mlsanity.com/paper/1803.08488/) (2018)
- [Characterising and Detecting Sponsored Influencer Posts on Instagram](https://mlsanity.com/paper/2011.05757/) (2020)
- [Pluggable Social Artificial Intelligence for Enabling Human-Agent Teaming](https://mlsanity.com/paper/1909.04492/) (2019)

## Built on by (most cited first, 9 of 9)

- [Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges](https://mlsanity.com/paper/2403.02990/) (2024)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [Across Platforms and Languages: Dutch Influencers and Legal Disclosures on Instagram, YouTube and TikTok](https://mlsanity.com/paper/2407.12451/) (2024)
- [Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Dataset](https://mlsanity.com/paper/2410.09416/) (2024)
- [InstaSynth: Opportunities and Challenges in Generating Synthetic Instagram Data with ChatGPT for Sponsored Content Detection](https://mlsanity.com/paper/2403.15214/) (2024)
- [Evaluating LLM-Generated Legal Explanations for Regulatory Compliance in Social Media Influencer Marketing](https://mlsanity.com/paper/2510.08111/) (2025)
- [ChatGPT and U(X): A Rapid Review on Measuring the User Experience](https://mlsanity.com/paper/2503.15808/) (2025)
- [Towards High-Fidelity Synthetic Multi-platform Social Media Datasets via Large Language Models](https://mlsanity.com/paper/2505.02858/) (2025)
- [ChildSafeAds Shared Task 2026: Commercial Content in Child-Facing YouTube Videos](https://mlsanity.com/paper/2608.19165/) (2026)

## Abstract (arXiv)

Regulatory bodies worldwide are intensifying their efforts to ensure transparency in influencer marketing on social media through instruments like the Unfair Commercial Practices Directive (UCPD) in the European Union, or Section 5 of the Federal Trade Commission Act. Yet enforcing these obligations has proven to be highly problematic due to the sheer scale of the influencer market. The task of automatically detecting sponsored content aims to enable the monitoring and enforcement of such regulations at scale. Current research in this field primarily frames this problem as a machine learning task, focusing on developing models that achieve high classification performance in detecting ads. These machine learning tasks rely on human data annotation to provide ground truth information. However, agreement between annotators is often low, leading to inconsistent labels that hinder the reliability of models. To improve annotation accuracy and, thus, the detection of sponsored content, we propose using chatGPT to augment the annotation process with phrases identified as relevant features and brief explanations. Our experiments show that this approach consistently improves inter-annotator agreement and annotation accuracy. Additionally, our survey of user experience in the annotation task indicates that the explanations improve the annotators' confidence and streamline the process. Our proposed methods can ultimately lead to more transparency and alignment with regulatory requirements in sponsored content detection.