mlsanityData to 14 Oct 2026

Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task

arXiv 1711.01505 ↗First posted 4 Nov 2017cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

This paper presents a summary of the first Workshop on Building Linguistically Generalizable Natural Language Processing Systems, and the associated Build It Break It, The Language Edition shared task. The goal of this workshop was to bring together researchers in NLP and linguistics with a shared task aimed at testing the generalizability of NLP systems beyond the distributions of their training data. We describe the motivation, setup, and participation of the shared task, provide discussion of some highlighted results, and discuss lessons learned.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). First posted on arXiv 4 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 8; 2023: 5; 2024: 1; 2025: 0; 2026: 1.

24 papers.

Sort:

  1. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  2. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  3. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  4. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
  5. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  6. 2022Evaluation Gaps in Machine Learning Practicecited by 49
  7. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 31
  8. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  9. 2022From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible MLcited by 27
  10. 2021The King is Naked: on the Notion of Robustness for Natural Language Processingcited by 16
  11. 2022Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectorscited by 14
  12. 2022Metaethical Perspectives on 'Benchmarking' AI Ethicscited by 12
12 more
  1. 2023Verifying the Robustness of Automatic Credibility Assessmentcited by 8
  2. 2021DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluationcited by 5
  3. 2022Cheating Automatic Short Answer Grading: On the Adversarial Usage of Adjectives and Adverbscited by 5
  4. 2023Break it, Imitate it, Fix it: Robustness by Generating Human-Like Attackscited by 5
  5. 2022Event Coreference Resolution for Contentious Politics Eventscited by 4
  6. 2023A Multilingual Evaluation of NER Robustness to Adversarial Inputscited by 2
  7. 2021Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Taskscited by 1
  8. 2023On Degrees of Freedom in Defining and Testing Natural Language Understandingcited by 1
  9. 2024How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generationcited by 1
  10. 2022Using Word Embeddings to Analyze Protests Newscited by 0
  11. 2023Analyzing the Generalizability of Deep Contextualized Language Representations For Text Classificationcited by 0
  12. 2026Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling Taskscited by 0