A P=aninian Foundation for Indic Language Processing
Not on the map. Why?
Abstract (arXiv)
More than a billion people communicate in Indic languages, yet the natural language processing infrastructure serving them remains fragmented and underdeveloped. The cause is structural: the field organizes its tools and benchmarks around individual languages or small subsets of genealogical language families, building separate analyzers, parsers, and datasets for each language and starting over for the next. This overlooks a deep regularity. Through more than two millennia of convergence around Sanskrit, Indic languages came to share a morphosyntactic architecture formalized in P=anini's grammar, the Ast=adhy=ay=i. This cuts across genealogical lines, uniting languages through a common framework. We argue that this P=aninian framework supplies a unifying computational architecture the field has lacked, and that benchmarks grounded explicitly in it would make Indic language systems more accurate, more data-efficient, and more transferable, effectively merging many apparently disparate and sparse Indic language resources into a single high-resource metalanguage bedrock. We propose a four-part benchmark suite to render this shared architecture explicit, measurable, and ready to be leveraged for practical applications. Moreover, we underscore the question it raises for interpretability research: whether neural models trained on these languages come to represent P=anini's categories on their own.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 58 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
13 of 58 references.
References · 13
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
- 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 59
- 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
- 2024INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languagescited by 24
- 2024One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Taskscited by 15
- 2025Tokenization Matters: Improving Zero-Shot NER for Indic Languagescited by 12
- 2025MorphTok: Morphologically Grounded Tokenization for Indian Languagescited by 7
- 2023Cross-lingual Transfer in Programming Languages: An Extensive Empirical Studycited by 5
- 2024LEVOS: Leveraging Vocabulary Overlap with Sanskrit to Generate Technical Lexicons in Indian Languagescited by 4
- 2025Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphologycited by 3
- 2020Sanskrit Segmentation Revisitedcited by 2
1 more
Built on by
Not cited yet.