mlsanityData to 7 Oct 2026

GreaseLM: Graph REASoning Enhanced Language Models for Question Answering

arXiv 2201.08860 ↗First posted 21 Jan 2022cs.CL cs.LGcited by 170

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it. However, pretrained language models (LM), the foundation of most modern QA systems, do not robustly represent latent relationships between concepts, which is necessary for reasoning. While knowledge graphs (KG) are often used to augment LMs with structured representations of world knowledge, it remains an open question how to effectively fuse and reason over the KG representations and the language context, which provides situational constraints and nuances. In this work, we propose GreaseLM, a new model that fuses encoded representations from pretrained LMs and graph neural networks over multiple layers of modality interaction operations. Information from both modalities propagates to the other, allowing language context representations to be grounded by structured world knowledge, and allowing linguistic nuances (e.g., negation, hedging) in the context to inform the graph representations of knowledge. Our results on three benchmarks in the commonsense reasoning (i.e., CommonsenseQA, OpenbookQA) and medical question answering (i.e., MedQA-USMLE) domains demonstrate that GreaseLM can more reliably answer questions that require reasoning over both situational constraints and structured knowledge, even outperforming models 8x larger.

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 49 references that are in the map. First posted on arXiv 21 Jan 2022.

Builds on

39 of 49 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017Graph Attention Networkscited by 8,854
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  12. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
27 more
  1. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  2. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  3. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  4. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  5. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  6. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  7. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  8. 2017A simple neural network module for relational reasoningcited by 345
  9. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  10. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  11. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  12. 2018Deep Learning: A Critical Appraisalcited by 214
  13. 2020Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddingscited by 169
  14. 2020Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphscited by 128
  15. 2020Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answeringcited by 122
  16. 2020Towards Controllable Biases in Language Generationcited by 112
  17. 2020CoLAKE: Contextualized Language and Knowledge Embeddingcited by 86
  18. 2020JAKET: Joint Pre-training of Knowledge Graph and Language Understandingcited by 73
  19. 2019Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answeringcited by 68
  20. 2018Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledgecited by 48
  21. 2018Improving Natural Language Inference Using External Knowledge in the Science Questions Domaincited by 48
  22. 2020Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answeringcited by 44
  23. 2019From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Projectcited by 33
  24. 2021Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resourcescited by 30
  25. 2020Exploiting Structured Knowledge in Text via Graph-Guided Representation Learningcited by 26
  26. 2020Learning Contextualized Knowledge Structures for Commonsense Reasoningcited by 18
  27. 2021Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answeringcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 25; 2023: 50; 2024: 50; 2025: 28; 2026: 17.

170 papers.

Sort:

By area:

  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  2. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  3. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  4. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  5. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  6. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  7. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  8. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  9. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  10. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  11. 2023Large Language Models as Analogical Reasonerscited by 107
  12. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
38 more
  1. 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
  2. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  3. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  4. 2024A Survey of Large Language Models for Graphscited by 71
  5. 2023Graph Neural Prompting with Large Language Modelscited by 64
  6. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasonerscited by 63
  7. 2023Natural Language Reasoning, A Surveycited by 61
  8. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoningcited by 60
  9. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56
  10. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  11. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  12. 2023Learning to Generate Better Than Your LLMcited by 49
  13. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  14. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  15. 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
  16. 2022Retrieval-based Controllable Molecule Generationcited by 44
  17. 2022Knowledge Graph Embedding: A Survey from the Perspective of Representation Spacescited by 41
  18. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphscited by 35
  19. 2023Graph Reasoning for Question Answering with Triplet Retrievalcited by 34
  20. 2023ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddingscited by 32
  21. 2022A Survey of Knowledge-Intensive NLP with Pre-Trained Language Modelscited by 30
  22. 2023KnowGPT: Knowledge Graph based Prompting for Large Language Modelscited by 30
  23. 2024Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancementcited by 30
  24. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 30
  25. 2023Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphscited by 29
  26. 2022Neuro-Symbolic Procedural Planning with Commonsense Promptingcited by 28
  27. 2023Large Language Models as Topological Structure Enhancers for Text-Attributed Graphscited by 27
  28. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  29. 2024Graph-Based Retriever Captures the Long Tail of Biomedical Knowledgecited by 20
  30. 2024Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspectivecited by 20
  31. 2024InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integrationcited by 19
  32. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuningcited by 19
  33. 2024Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Modelscited by 18
  34. 2023GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answeringcited by 17
  35. 2024Can Language Models Act as Knowledge Bases at Scale?cited by 17
  36. 2025Graph4MM: Weaving Multimodal Learning with Structural Informationcited by 17
  37. 2024Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrievalcited by 16
  38. 2022BIC: Twitter Bot Detection with Text-Graph Interaction and Semantic Consistencycited by 14

50 most cited of 170 All 170 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering
  2. 2023ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph
  3. 2025Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering
  4. 2024G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning
  5. 2025ARK-V1: An LLM-Agent for Knowledge Graph Question Answering Requiring Commonsense Reasoning
  6. 2026From Chains to Graphs: Self-Structured Reasoning for General-Domain LLMs
  7. 2026Leveraging LLM-GNN Integration for Open-World Question Answering over Knowledge Graphs
  8. 2025iQUEST: An Iterative Question-Guided Framework for Knowledge Base Question Answering