# Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection

arXiv: https://arxiv.org/abs/2506.00955 · HTML: https://mlsanity.com/paper/2506.00955/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 1 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation](https://mlsanity.com/paper/2407.05361/) (2024)
- [Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)](https://mlsanity.com/paper/1906.01815/) (2019)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [Is ChatGPT Equipped with Emotional Dialogue Capabilities?](https://mlsanity.com/paper/2304.09582/) (2023)
- [The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks](https://mlsanity.com/paper/2304.13861/) (2023)
- [A Multimodal Corpus for Emotion Recognition in Sarcasm](https://mlsanity.com/paper/2206.02119/) (2022)
- [SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding](https://mlsanity.com/paper/2408.11319/) (2024)
- [Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models](https://mlsanity.com/paper/2308.11578/) (2023)
- [AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data](https://mlsanity.com/paper/2309.13905/) (2023)
- [Can Pre-trained Language Models Understand Chinese Humor?](https://mlsanity.com/paper/2407.04105/) (2024)
- [From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs](https://mlsanity.com/paper/2408.17026/) (2024)
- [A Functional Trade-off between Prosodic and Semantic Cues in Conveying Sarcasm](https://mlsanity.com/paper/2408.14892/) (2024)
- [AMuSeD: An Attentive Deep Neural Network for Multimodal Sarcasm Detection Incorporating Bi-modal Data Augmentation](https://mlsanity.com/paper/2412.10103/) (2024)

## Built on by (most cited first, 4 of 4)

- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis](https://mlsanity.com/paper/2508.13028/) (2025)
- [When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms](https://mlsanity.com/paper/2604.16767/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)

## Abstract (arXiv)

Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity. In addition, existing detection systems often rely on multimodal data, limiting their applicability in contexts where only speech is available. To address this, we propose an annotation pipeline that leverages large language models (LLMs) to generate a sarcasm dataset. Using a publicly available sarcasm-focused podcast, we employ GPT-4o and LLaMA 3 for initial sarcasm annotations, followed by human verification to resolve disagreements. We validate this approach by comparing annotation quality and detection performance on a publicly available sarcasm dataset using a collaborative gating architecture. Finally, we introduce PodSarc, a large-scale sarcastic speech dataset created through this pipeline. The detection model achieves a 73.63% F1 score, demonstrating the dataset's potential as a benchmark for sarcasm detection research.