mlsanityData to 7 Oct 2026

Discontinuous Constituency and BERT: A Case Study of Dutch

arXiv 2203.01063 ↗First posted 2 Mar 2022cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 12 of its 37 references that are in the map. First posted on arXiv 2 Mar 2022.

Builds on

12 of 37 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2015Gated Graph Sequence Neural Networkscited by 792
  6. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 119
  7. 2019BERTje: A Dutch BERT Modelcited by 109
  8. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88
  9. 2021Probing BERT in Hyperbolic Spacescited by 50
  10. 2021SICKNL: A Dataset for Dutch Natural Language Inferencecited by 20
  11. 2020Parsing as Pretrainingcited by 19
  12. 2019The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviewscited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2.

3 papers.

  1. 2022Word-order typology in Multilingual BERT: A case study in subordinate-clause detectioncited by 3
  2. 2023Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativizationcited by 2
  3. 2023Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languagescited by 2