mlsanityData to 7 Oct 2026

DynaSent: A Dynamic Benchmark for Sentiment Analysis

arXiv 2012.15349 ↗First posted 30 Dec 2020cs.CLcited by 63

Not on the map. Why?

Abstract (arXiv)

We introduce DynaSent ('Dynamic Sentiment'), a new English-language benchmark task for ternary (positive/negative/neutral) sentiment analysis. DynaSent combines naturally occurring sentences with sentences created using the open-source Dynabench Platform, which facilities human-and-model-in-the-loop dataset creation. DynaSent has a total of 121,634 sentences, each validated by five crowdworkers, and its development and test splits are designed to produce chance performance for even the best models we have been able to develop; when future models solve this task, we will use them to create DynaSent version 2, continuing the dynamic evolution of this benchmark. Here, we report on the dataset creation effort, focusing on the steps we took to increase quality and reduce artifacts. We also present evidence that DynaSent's Neutral category is more coherent than the comparable category in other benchmarks, and we motivate training models from scratch for each round over successive fine-tuning.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 10; 2023: 14; 2024: 15; 2025: 6; 2026: 2.

63 papers.

Sort:

By area:

  1. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  2. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  3. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  4. 2022RealTime QA: What's the Answer Right Now?cited by 239
  5. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  6. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  7. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  8. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  9. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  10. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  11. 2022NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysiscited by 64
  12. 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
38 more
  1. 2023How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Taskscited by 58
  2. 2021Human-Adversarial Visual Question Answeringcited by 57
  3. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  4. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  5. 2023Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Informationcited by 52
  6. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  7. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  8. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  9. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 30
  10. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  11. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 23
  12. 2023What's the Meaning of Superhuman Performance in Today's NLU?cited by 22
  13. 2024Understanding Linear Probing then Fine-tuning Language Models from NTK Perspectivecited by 21
  14. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  15. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  16. 2024Improving Model Evaluation using SMART Filtering of Benchmark Datasetscited by 20
  17. 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
  18. 2023Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collectioncited by 17
  19. 2022Dynatask: A Framework for Creating Dynamic AI Benchmark Taskscited by 16
  20. 2022A Theory of Dynamic Benchmarkscited by 15
  21. 2023Being Right for Whose Right Reasons?cited by 12
  22. 2023Rather a Nurse than a Physician -- Contrastive Explanations under Investigationcited by 11
  23. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolboxcited by 11
  24. 2024Confidence Calibration of Classifiers with Many Classescited by 11
  25. 2021Clean or Annotate: How to Spend a Limited Data Collection Budgetcited by 10
  26. 2024Efficient Lifelong Model Evaluation in an Era of Rapid Progresscited by 10
  27. 2022longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attackscited by 8
  28. 2022Benchmarking Long-tail Generalization with Likelihood Splitscited by 8
  29. 2024Improving Black-box Robustness with In-Context Rewritingcited by 8
  30. 2021Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarkingcited by 7
  31. 2023Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Modelscited by 7
  32. 2025Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymizationcited by 4
  33. 2025Eliciting Behaviors in Multi-Turn Conversationscited by 4
  34. 2021Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classificationcited by 3
  35. 2022Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculationcited by 3
  36. 2023RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Trainingcited by 3
  37. 2024Co-training for Low Resource Scientific Natural Language Inferencecited by 3
  38. 2025BRoverbs -- Measuring how much LLMs understand Portuguese proverbscited by 3

50 most cited of 63 All 63 (JSON)