mlsanityData to 7 Oct 2026

Revealing Trends in Datasets from the 2022 ACL and EMNLP Conferences

arXiv 2404.08666 ↗First posted 31 Mar 2024cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Natural language processing (NLP) has grown significantly since the advent of the Transformer architecture. Transformers have given birth to pre-trained large language models (PLMs). There has been tremendous improvement in the performance of NLP systems across several tasks. NLP systems are on par or, in some cases, better than humans at accomplishing specific tasks. However, it remains the norm that better quality datasets at the time of pretraining enable PLMs to achieve better performance, regardless of the task. The need to have quality datasets has prompted NLP researchers to continue creating new datasets to satisfy particular needs. For example, the two top NLP conferences, ACL and EMNLP, accepted ninety-two papers in 2022, introducing new datasets. This work aims to uncover the trends and insights mined within these datasets. Moreover, we provide valuable suggestions to researchers interested in curating datasets in the future.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 61 of its 95 references that are in the map. First posted on arXiv 31 Mar 2024.

Builds on

61 of 95 references.

References · 61

  1. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  2. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  3. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  4. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  5. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  6. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  7. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  8. 2022Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehensioncited by 79
  9. 2020KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Basecited by 76
  10. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  11. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  12. 2022MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extractioncited by 65
38 more
  1. 2022ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcriptscited by 59
  2. 2022SQuALITY: Building a Long-Document Summarization Dataset the Hard Waycited by 50
  3. 2021MReD: A Meta-Review Dataset for Structure-Controllable Text Generationcited by 42
  4. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  5. 2021A Statutory Article Retrieval Dataset in Frenchcited by 40
  6. 2022Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Counteringcited by 40
  7. 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 39
  8. 2022CICERO: A Dataset for Contextualized Commonsense Inference in Dialoguescited by 38
  9. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  10. 2022MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translationcited by 38
  11. 2022D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chatcited by 37
  12. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  13. 2022LEVEN: A Large-Scale Chinese Legal Event Detection Datasetcited by 36
  14. 2022WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Typescited by 33
  15. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29
  16. 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
  17. 2022Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Taskscited by 28
  18. 2022M2D2: A Massively Multi-domain Language Modeling Datasetcited by 26
  19. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  20. 2022IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Taskscited by 23
  21. 2023Long Text and Multi-Table Summarization: Dataset and Methodcited by 23
  22. 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 21
  23. 2022TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languagescited by 21
  24. 2022CrossRE: A Cross-Domain Dataset for Relation Extractioncited by 20
  25. 2022MEE: A Novel Multilingual Event Extraction Datasetcited by 17
  26. 2022MSCTD: A Multimodal Sentiment Chat Translation Datasetcited by 15
  27. 2022BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examplescited by 15
  28. 2022Visual Named Entity Linking: A New Dataset and A Baselinecited by 15
  29. 2022KOLD: Korean Offensive Language Datasetcited by 14
  30. 2022Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Datasetcited by 14
  31. 2022BotsTalk: Machine-sourced Framework for Automatic Curation of Large-scale Multi-skill Dialogue Datasetscited by 13
  32. 2022A Dataset for Hyper-Relational Extraction and a Cube-Filling Approachcited by 13
  33. 2022Leveraging QA Datasets to Improve Generative Data Augmentationcited by 12
  34. 2022DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languagescited by 11
  35. 2022JamPatoisNLI: A Jamaican Patois Natural Language Inference Datasetcited by 11
  36. 2022Augmenting Multi-Turn Text-to-SQL Datasets with Self-Playcited by 9
  37. 2022Questioning the Validity of Summarization Datasets and Improving Their Factual Consistencycited by 9
  38. 2022A Benchmark and Dataset for Post-OCR text correction in Sanskritcited by 9

50 most cited of 61 All 61 (JSON)

Built on by

Not cited yet.