# Gyan: An Explainable Neuro-Symbolic Language Model

arXiv: https://arxiv.org/abs/2605.04759 · HTML: https://mlsanity.com/paper/2605.04759/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation](https://mlsanity.com/paper/1911.06136/) (2019)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [Testing AI on language comprehension tasks reveals insensitivity to underlying meaning](https://mlsanity.com/paper/2302.12313/) (2023)
- [From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning](https://mlsanity.com/paper/2505.17117/) (2025)
- [Text Summarization using Abstract Meaning Representation](https://mlsanity.com/paper/1706.01678/) (2017)
- [Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings](https://mlsanity.com/paper/1603.06067/) (2016)
- [Embedding Words and Senses Together via Joint Knowledge-Enhanced Training](https://mlsanity.com/paper/1612.02703/) (2016)
- [What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris](https://mlsanity.com/paper/2205.07750/) (2022)
- [Contextual Word Representations: A Contextual Introduction](https://mlsanity.com/paper/1902.06006/) (2019)

## Abstract (arXiv)

Transformer based pre-trained large language models have become ubiquitous. There is increasing evidence to suggest that even with large scale pre-training, these models do not capture complete compositional context and certainly not, the full human analogous context. Besides, by the very nature of the architecture, these models hallucinate, are difficult to maintain, are not easily interpretable and require enormous compute resources for training and inference. Here, we describe Gyan, an explainable language model based on a novel non-transformer architecture, without any of these limitations. Gyan achieves SOTA performance on 3 widely cited data sets and superior performance on two proprietary data sets. The novel architecture decouples the language model from knowledge acquisition and representation. The model draws on rhetorical structure theory, semantic role theory and knowledge-based computational linguistics. Gyan's meaning representation structure captures the complete compositional context and attempts to mimic humans by expanding the context to a 'world model'. AI model adoption critically depends on trust and transparency especially in mission critical use cases. Collectively, our results demonstrate that it is possible to create models which are trustable and reliable for mission critical tasks. We believe our work has tremendous potential for guiding the development of transparent and trusted architectures for language models.