mlsanityData to 7 Oct 2026

DataCLUE: A Benchmark Suite for Data-centric NLP

arXiv 2111.08647 ↗First posted 16 Nov 2021cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Data-centric AI has recently proven to be more effective and high-performance, while traditional model-centric AI delivers fewer and fewer benefits. It emphasizes improving the quality of datasets to achieve better model performance. This field has significant potential because of its great practicability and getting more and more attention. However, we have not seen significant research progress in this field, especially in NLP. We propose DataCLUE, which is the first Data-Centric benchmark applied in NLP field. We also provide three simple but effective baselines to foster research in this field (improve Macro-F1 up to 5.7% point). In addition, we conduct comprehensive experiments with human annotators and show the hardness of DataCLUE. We also try an advanced method: the forgetting informed bootstrapping label correction method. All the resources related to DataCLUE, including datasets, toolkit, leaderboard, and baselines, is available online at https://github.com/CLUEbenchmark/DataCLUE

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

12 of 18 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  5. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  6. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  7. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  8. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
  9. 2019Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approachescited by 181
  10. 2018Learning to Learn from Noisy Labeled Datacited by 160
  11. 2020ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networkscited by 33
  12. 2021Data-Centric AI Requires Rethinking Data Notioncited by 8

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 2.

6 papers.

  1. 2022Handling and Presenting Harmful Text in NLP Researchcited by 43
  2. 2022Detecting Label Errors by using Pre-Trained Language Modelscited by 16
  3. 2022Statistical Dataset Evaluation: Reliability, Difficulty, and Validitycited by 2
  4. 2023A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasetscited by 2
  5. 2022Active-Learning-as-a-Service: An Automatic and Efficient MLOps System for Data-Centric AIcited by 0
  6. 2023DataCI: A Platform for Data-Centric AI on Streaming Datacited by 0