Concept-Grounded Attention: A Controlled Evaluation of Graph-Injected Attention, Temporal Versioning, and Epistemic Status
Not on the map. Why?
Abstract (arXiv)
Knowledge-intensive language-model systems typically represent external knowledge as text chunks or static graphs, with limited support for concept evolution, point-in-time reasoning, and distinctions between validated and inferred knowledge. We introduce the Concept Lifecycle Model (CLM), which represents concepts as persistent, graph-grounded, temporally versioned entities with explicit provenance and epistemic status, and Concept-Grounded Attention (CGA), which injects concept-graph structure into transformer computation through graph-biased self-attention (Form A) and gated cross-attention over concept nodes (Form B). We evaluate the framework in controlled settings using disabled-mechanism baselines. On 200 MuSiQue and HotpotQA questions with retrieval fixed, concept-graph retrieval recovers explicit multi-hop paths but does not improve evidence recall. Form A appears to steer attention, with 2.76 times more attention on gold than distractor concepts, but the same ratio occurs when Form A is disabled; the learned bias is negligible and no answers change. An identity-preserving Form B improves F1 from 0.188 to 0.221, but control concepts yield 0.213, indicating that most of the gain reflects added capacity. On LongMemEval, explicit temporal representation improves answer accuracy by 13 to 25 points across all tested generators, up to 122B parameters, while simplified CLM version resolution performs similarly to dated serialization because concept identity is not established reliably. On a synthetic source-independence task, protocol-derived epistemic status reduces unsupported assertions from 28% to 0.1% in a fine-tuned small model and from 19-68% to 0-5% in 72-122B models. Overall, the results support making temporal validity and epistemic status explicit, while showing that graph-attention diagnostics are not informative without disabled-mechanism controls.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
24 of 35 references.
References · 24
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2019Attention is not Explanationcited by 916
- 2016Complex Embeddings for Simple Link Predictioncited by 897
- 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2019Attention is not not Explanationcited by 629
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
- 2019Is Attention Interpretable?cited by 387
12 more
- 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
- 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
- 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
- 2019Knowledge Enhanced Contextual Word Representationscited by 294
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
- 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
- 2022Pure Transformers are Powerful Graph Learnerscited by 188
- 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 170
- 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
- 2020Tensor Decompositions for temporal knowledge base completioncited by 115
- 2024StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurizationcited by 70
- 2026STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentationcited by 1
Built on by
Not cited yet.