mlsanityData to 7 Oct 2026

Word-order typology in Multilingual BERT: A case study in subordinate-clause detection

arXiv 2205.11987 ↗First posted 24 May 2022cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The capabilities and limitations of BERT and similar models are still unclear when it comes to learning syntactic abstractions, in particular across languages. In this paper, we use the task of subordinate-clause detection within and across languages to probe these properties. We show that this task is deceptively simple, with easy gains offset by a long tail of harder cases, and that BERT's zero-shot performance is dominated by word-order effects, mirroring the SVO/VSO/SOV typology.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 15 of its 43 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

15 of 43 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  3. 2019How multilingual is Multilingual BERT?cited by 642
  4. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  5. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  6. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 119
  7. 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 108
  8. 2019Is Multilingual BERT Fluent in Language Generation?cited by 28
  9. 2020A Survey on Text Simplificationcited by 13
  10. 2021Does Typological Blinding Impede Cross-Lingual Sharing?cited by 9
  11. 2019Low-Resource Syntactic Transfer with Unsupervised Source Reorderingcited by 8
  12. 2017Found in Translation: Reconstructing Phylogenetic Language Trees from Translationscited by 6
3 more
  1. 2021On the Relation between Syntactic Divergence and Zero-Shot Performancecited by 4
  2. 2022Discontinuous Constituency and BERT: A Case Study of Dutchcited by 3
  3. 2022Can pre-trained Transformers be used in detecting complex sensitive sentences? -- A Monsanto case studycited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2024Multilingual Gradient Word-Order Typology from Universal Dependenciescited by 9
  2. 2023Exploring Linguistic Properties of Monolingual BERTs with Typological Classification among Languagescited by 6
  3. 2023Improving Cross-Lingual Transfer through Subtree-Aware Word Reorderingcited by 4