# Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning

arXiv: https://arxiv.org/abs/2509.16422 · HTML: https://mlsanity.com/paper/2509.16422/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 16 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 2 of 2)

- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [CxGBERT: BERT meets Construction Grammar](https://mlsanity.com/paper/2011.04134/) (2020)

## Built on by (most cited first, 1 of 1)

- [CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models](https://mlsanity.com/paper/2602.21978/) (2026)

## Abstract (arXiv)

We probe large language models' ability to learn deep form-meaning mappings as defined by construction grammars. We introduce the ConTest-NLI benchmark of 80k sentences covering eight English constructions from highly lexicalized to highly schematic. Our pipeline generates diverse synthetic NLI triples via templating and the application of a model-in-the-loop filter. This provides aspects of human validation to ensure challenge and label reliability. Zero-shot tests on leading LLMs reveal a 24% drop in accuracy between naturalistic (88%) and adversarial data (64%), with schematic patterns proving hardest. Fine-tuning on a subset of ConTest-NLI yields up to 9% improvement, yet our results highlight persistent abstraction gaps in current LLMs and offer a scalable framework for evaluating construction-informed learning.