mlsanityData to 7 Oct 2026

Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

arXiv 2203.07285 ↗First posted 14 Mar 2022cs.CL cs.AIcited by 19

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Generative commonsense reasoning (GCR) in natural language is to reason about the commonsense while generating coherent text. Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. Nevertheless, these approaches have seldom investigated diversity in the GCR tasks, which aims to generate alternative explanations for a real-world situation or predict all possible outcomes. Diversifying GCR is challenging as it expects to generate multiple outputs that are not only semantically different but also grounded in commonsense knowledge. In this paper, we propose MoKGE, a novel method that diversifies the generative reasoning by a mixture of expert (MoE) strategy on commonsense knowledge graphs (KG). A set of knowledge experts seek diverse reasoning on KG to encourage various generation outputs. Empirical experiments demonstrated that MoKGE can significantly improve the diversity while achieving on par performance on accuracy on two GCR benchmarks, based on both automatic and human evaluations.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 33 of its 44 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

33 of 44 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  10. 2019Composition-based Multi-Relational Graph Convolutional Networkscited by 395
  11. 2019Abductive Commonsense Reasoningcited by 300
  12. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
21 more
  1. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  2. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  3. 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
  4. 2020A Knowledge-Enhanced Pretraining Model for Commonsense Story Generationcited by 116
  5. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
  6. 2020KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoningcited by 106
  7. 2019Mixture Models for Diverse Machine Translation: Tricks of the Tradecited by 77
  8. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 66
  9. 2019Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphscited by 59
  10. 2020Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graphcited by 57
  11. 2017A Deep Generative Framework for Paraphrase Generationcited by 54
  12. 2019Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanationcited by 43
  13. 2020SemEval-2020 Task 4: Commonsense Validation and Explanationcited by 42
  14. 2018Toward Diverse Text Generation with Inverse Reinforcement Learningcited by 41
  15. 2021Dict-BERT: Enhancing Language Model Pre-training with Dictionarycited by 38
  16. 2019Mixture Content Selection for Diverse Sequence Generationcited by 27
  17. 2021Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representationscited by 19
  18. 2020Injecting Entity Types into Entity-Guided Text Generationcited by 7
  19. 2020Target Conditioning for One-to-Many Generationcited by 6
  20. 2021MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversationscited by 6
  21. 2021Sentence-Permuted Paragraph Generationcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 7; 2024: 2; 2025: 4.

19 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  3. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  4. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  5. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  6. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  7. 2024InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integrationcited by 19
  8. 2024Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learningcited by 19
  9. 2023Generating Visual Spatial Description via Holistic 3D Scene Understandingcited by 16
  10. 2023Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregationcited by 16
  11. 2022Retrieval Augmentation for Commonsense Reasoning: A Unified Approachcited by 14
  12. 2025Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Abilitycited by 10
7 more
  1. 2025Mosaic Pruning: A Hierarchical Framework for Generalizable Pruning of Mixture-of-Experts Modelscited by 10
  2. 2023CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured Knowledge Aggregationcited by 8
  3. 2023R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQLcited by 7
  4. 2022Dense-ATOMIC: Towards Densely-connected ATOMIC with High Knowledge Coverage and Massive Multi-hop Pathscited by 6
  5. 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
  6. 2025RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Modelscited by 1
  7. 2022Very Large Language Model as a Unified Methodology of Text Miningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Knowledge Graph-Augmented Korean Generative Commonsense Reasoning
  2. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey
  3. 2022Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
  4. 2026Synthetic Data Generation for Training Diversified Commonsense Reasoning Models
  5. 2022DimonGen: Diversified Generative Commonsense Reasoning for Explaining Concept Relationships
  6. 2024The Mystery of Compositional Generalization in Graph-based Generative Commonsense Reasoning
  7. 2023SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoning
  8. 2024Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs