# Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts

arXiv: https://arxiv.org/abs/2505.24427 · HTML: https://mlsanity.com/paper/2505.24427/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Prediction](https://mlsanity.com/paper/2206.05238/) (2022)
- [iSarcasm: A Dataset of Intended Sarcasm](https://mlsanity.com/paper/1911.03123/) (2019)
- [A Review of Generalizability and Transportability](https://mlsanity.com/paper/2102.11904/) (2021)
- [Crowdsourcing and Validating Event-focused Emotion Corpora for German and English](https://mlsanity.com/paper/1905.13618/) (2019)
- [Multimodal Emotion Classification](https://mlsanity.com/paper/1903.12520/) (2019)
- [Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options](https://mlsanity.com/paper/2010.06122/) (2020)
- [APPReddit: a Corpus of Reddit Posts Annotated for Appraisal](https://mlsanity.com/paper/2205.15627/) (2022)
- [Principles from Clinical Research for NLP Model Generalization](https://mlsanity.com/paper/2311.03663/) (2023)
- [Can Factual Statements be Deceptive? The DeFaBel Corpus of Belief-based Deception](https://mlsanity.com/paper/2403.10185/) (2024)
- [Dealing with Controversy: An Emotion and Coping Strategy Corpus Based on Role Playing](https://mlsanity.com/paper/2409.19025/) (2024)
- [Expressions Causing Differences in Emotion Recognition in Social Networking Service Documents](https://mlsanity.com/paper/2208.14244/) (2022)

## Abstract (arXiv)

Accurate modeling of subjective phenomena such as emotion expression requires data annotated with authors' intentions. Commonly such data is collected by asking study participants to donate and label genuine content produced in the real world, or create content fitting particular labels during the study. Asking participants to create content is often simpler to implement and presents fewer risks to participant privacy than data donation. However, it is unclear if and how study-created content may differ from genuine content, and how differences may impact models. We collect study-created and genuine multimodal social media posts labeled for emotion and compare them on several dimensions, including model performance. We find that compared to genuine posts, study-created posts are longer, rely more on their text and less on their images for emotion expression, and focus more on emotion-prototypical events. The samples of participants willing to donate versus create posts are demographically different. Study-created data is valuable to train models that generalize well to genuine data, but realistic effectiveness estimates require genuine data.