mlsanityData to 7 Oct 2026

Towards Corpus-Grounded Agentic LLMs for Multilingual Grammatical Analysis

arXiv 2512.00214 ↗First posted 28 Nov 2025cs.CLcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Empirical grammar research has become increasingly data-driven, but the systematic analysis of annotated corpora still requires substantial methodological and technical effort. We explore how agentic large language models (LLMs) can streamline this process by reasoning over annotated corpora and producing interpretable, data-grounded answers to linguistic questions. We introduce an agentic framework for corpus-grounded grammatical analysis that integrates concepts such as natural-language task interpretation, code generation, and data-driven reasoning. As a proof of concept, we apply it to Universal Dependencies (UD) corpora, testing it on multilingual grammatical tasks inspired by the World Atlas of Language Structures (WALS). The evaluation spans 13 word-order features and over 170 languages, assessing system performance across three complementary dimensions - dominant-order accuracy, order-coverage completeness, and distributional fidelity - which reflect how well the system generalizes, identifies, and quantifies word-order variations. The results demonstrate the feasibility of combining LLM reasoning with structured linguistic data, offering a first step toward interpretable, scalable automation of corpus-based grammatical inquiry.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 11 of its 33 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

11 of 33 references.

References · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 79
  5. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 65
  6. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineeringcited by 26
  7. 2023Large Linguistic Models: Investigating LLMs' metalinguistic abilitiescited by 18
  8. 2024Multilingual Gradient Word-Order Typology from Universal Dependenciescited by 9
  9. 2024Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learningcited by 8
  10. 2024Sparse Logistic Regression with High-order Features for Automatic Grammar Rule Extraction from Treebankscited by 2
  11. 2024From Text to CQL: Bridging Natural Language and Corpus Search Enginecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Agent-Driven Corpus Linguistics: A Framework for Autonomous Linguistic Discoverycited by 1
  2. 2026Evaluating Metalinguistic Knowledge in Large Language Models across the World's Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs
  2. 2025LingGym: How Far Are LLMs from Thinking Like Field Linguists?
  3. 2026LLM Agents as Computational Typologists
  4. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
  5. 2025SR-LLM: Rethinking the Structured Representation in Large Language Model
  6. 2025The Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in LLMs with Camlang
  7. 2026Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations
  8. 2025Creating ConLangs to Probe the Metalinguistic Grammatical Knowledge of LLMs