mlsanityData to 7 Oct 2026

SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research

arXiv 2310.14757 ↗First posted 23 Oct 2023cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Despite its relevance, the maturity of NLP for social media pales in comparison with general-purpose models, metrics and benchmarks. This fragmented landscape makes it hard for the community to know, for instance, given a task, which is the best performing model and how it compares with others. To alleviate this issue, we introduce a unified benchmark for NLP evaluation in social media, SuperTweetEval, which includes a heterogeneous set of tasks and datasets combined, adapted and constructed from scratch. We benchmarked the performance of a wide range of models on SuperTweetEval and our results suggest that, despite the recent advances in language modelling, social media remains challenging.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 44 of its 89 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

44 of 89 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  10. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  11. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  12. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
32 more
  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  3. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  4. 2021MetaICL: Learning to Learn In Contextcited by 450
  5. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  6. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  7. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  8. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  9. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
  10. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
  11. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  12. 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
  13. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  14. 2019Multilingual and Multi-Aspect Hate Speech Analysiscited by 133
  15. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  16. 2021XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyondcited by 114
  17. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  18. 2019TWEETQA: A Social Media Focused Question Answering Datasetcited by 55
  19. 2020"Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samplescited by 48
  20. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  21. 2021Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detectioncited by 45
  22. 2020Detecting East Asian Prejudice on Social Mediacited by 40
  23. 2022Generative Language Models for Paragraph-Level Question Generationcited by 28
  24. 2022Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shiftscited by 27
  25. 2020Quantifying Intimacy in Languagecited by 25
  26. 2018Short-Term Meaning Shift: A Distributional Explorationcited by 23
  27. 2014Analysis of Named Entity Recognition and Linking for Tweetscited by 20
  28. 2022TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Mediacited by 19
  29. 2022SemEval 2023 Task 9: Multilingual Tweet Intimacy Analysiscited by 14
  30. 2021Mitigating Temporal-Drift: A Simple Approach to Keep NER Models Crispcited by 7
  31. 2017Are Emojis Predictable?cited by 6
  32. 2022TweetNERD -- End to End Entity Linking Benchmark for Tweetscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 1.

9 papers.

  1. 2025AI in Mental Health: Emotional and Sentiment Analysis of Large Language Models' Responses to Depression, Anxiety, and Stress Queriescited by 3
  2. 2025A Survey of Machine Learning Models and Datasets for the Multi-label Classification of Textual Hate Speech in Englishcited by 2
  3. 2024WiDe-analysis: Enabling One-click Content Moderation Analysis on Wikipedia's Articles for Deletioncited by 1
  4. 2024Causal Effects of Trigger Words in Social Media Discussions: A Large-Scale Case Study about UK Politics on Redditcited by 0
  5. 2024A Systematic Analysis on the Temporal Generalization of Language Models in Social Mediacited by 0
  6. 2024Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Modelscited by 0
  7. 2025Hatevolution: What Static Benchmarks Don't Tell Uscited by 0
  8. 2025Predicting Social Media User Actions: A Hybrid Approach for Common and Rare Behavior Prediction on Blueskycited by 0
  9. 2026Text Analytics Evaluation Framework: A Case Study on LLMs and Social Mediacited by 0