mlsanityData to 14 Oct 2026

Do Language Models Encode Knowledge of Linguistic Constraint Violations?

arXiv 2605.12055 ↗First posted 12 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) achieve strong linguistic performance, yet their internal mechanisms for producing these predictions remain unclear. We investigate the hypothesis that LLMs encode representations of linguistic constraint violations within their parameters, which are selectively activated when processing ungrammatical sentences. To test this, we use sparse autoencoders to decompose polysemantic activations into sparse, monosemantic features and recover candidates for violation-related features. We introduce a sensitivity score for identifying features that are preferentially activated on constraint-violated versus well-formed inputs, enabling unsupervised detection of potential violation-specific features. We further propose a conjunctive falsification framework with three criteria evaluated jointly. Overall, the results are negative in two respects: (1) the falsification criteria are not jointly satisfied across linguistic phenomena, and (2) no features are consistently shared across all categories. While some phenomena show partial evidence of selective causal structure, the overall pattern provides limited support for a unified set of grammatical violation detectors in current LMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

15 of 20 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2022Toy Models of Superpositioncited by 971
  5. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  6. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  7. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  8. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  9. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  10. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
  11. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  12. 2021How is BERT surprised? Layerwise detection of linguistic anomaliescited by 24
3 more
  1. 2024Unveiling Linguistic Regions in Large Language Modelscited by 22
  2. 2025What Can String Probability Tell Us About Grammaticality?cited by 16
  3. 2025LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encodercited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025In Machina N400: Pinpointing Where a Causal Language Model Detects Semantic Violations
  2. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?
  3. 2026Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders
  4. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
  5. 2025Large Language Model probabilities cannot distinguish between possible and impossible language
  6. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoders
  7. 2025Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders
  8. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds