# DaLAJ - a dataset for linguistic acceptability judgments for Swedish: Format, baseline, sharing

arXiv: https://arxiv.org/abs/2105.06681 · HTML: https://mlsanity.com/paper/2105.06681/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 5 of 5)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [How Furiously Can Colourless Green Ideas Sleep? Sentence Acceptability in Context](https://mlsanity.com/paper/2004.00881/) (2020)

## Built on by (most cited first, 16 of 16)

- [RuCoLA: Russian Corpus of Linguistic Acceptability](https://mlsanity.com/paper/2210.12814/) (2022)
- [RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs](https://mlsanity.com/paper/2406.19232/) (2024)
- [Acceptability Judgements via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2205.09630/) (2022)
- [Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus](https://mlsanity.com/paper/2109.12053/) (2021)
- [XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs](https://mlsanity.com/paper/2502.19737/) (2025)
- [Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations](https://mlsanity.com/paper/2109.14017/) (2021)
- [NoCoLA: The Norwegian Corpus of Linguistic Acceptability](https://mlsanity.com/paper/2306.07790/) (2023)
- [Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2411.07474/) (2024)
- [Minimal Pair-Based Evaluation of Code-Switching](https://mlsanity.com/paper/2506.01840/) (2025)
- [Distributional Data Augmentation Methods for Low Resource Language](https://mlsanity.com/paper/2309.04862/) (2023)
- [DaLA: Danish Linguistic Acceptability Evaluation Guided by Real World Errors](https://mlsanity.com/paper/2512.04799/) (2025)
- [Quantum Transfer Learning for Acceptability Judgements](https://mlsanity.com/paper/2401.07777/) (2024)
- [QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments](https://mlsanity.com/paper/2508.16867/) (2025)
- [QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs](https://mlsanity.com/paper/2509.25664/) (2025)
- [COLA-GEC: A Bidirectional Framework for Enhancing Grammatical Acceptability and Error Correction](https://mlsanity.com/paper/2507.11867/) (2025)
- [Targeted Syntactic Evaluation of Language Models on Georgian Case Alignment](https://mlsanity.com/paper/2602.10661/) (2026)

## Abstract (arXiv)

We present DaLAJ 1.0, a Dataset for Linguistic Acceptability Judgments for Swedish, comprising 9 596 sentences in its first version; and the initial experiment using it for the binary classification task. DaLAJ is based on the SweLL second language learner data, consisting of essays at different levels of proficiency. To make sure the dataset can be freely available despite the GDPR regulations, we have sentence-scrambled learner essays and removed part of the metadata about learners, keeping for each sentence only information about the mother tongue and the level of the course where the essay has been written. We use the normalized version of learner language as the basis for the DaLAJ sentences, and keep only one error per sentence. We repeat the same sentence for each individual correction tag used in the sentence. For DaLAJ 1.0 we have used four error categories (out of 35 available in SweLL), all connected to lexical or word-building choices. Our baseline results for the binary classification show an accuracy of 58% for DaLAJ 1.0 using BERT embeddings. The dataset is included in the SwedishGlue (Swe. SuperLim) benchmark. Below, we describe the format of the dataset, first experiments, our insights and the motivation for the chosen approach to data sharing.