# Retrieval Enhanced Model for Commonsense Generation

arXiv: https://arxiv.org/abs/2105.11174 · HTML: https://mlsanity.com/paper/2105.11174/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 24 May 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research](https://mlsanity.com/paper/1904.03493/) (2019)
- [SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference](https://mlsanity.com/paper/1808.05326/) (2018)
- [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020)
- [A Survey of Knowledge-Enhanced Text Generation](https://mlsanity.com/paper/2010.04389/) (2020)
- [KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning](https://mlsanity.com/paper/2009.12677/) (2020)
- [Pre-training Text-to-Text Transformers for Concept-centric Common Sense](https://mlsanity.com/paper/2011.07956/) (2020)
- [An Enhanced Knowledge Injection Model for Commonsense Generation](https://mlsanity.com/paper/2012.00366/) (2020)

## Built on by (most cited first, 18 of 18)

- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [Retrieval-Augmented Generation with Conflicting Evidence](https://mlsanity.com/paper/2504.13079/) (2025)
- [RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records](https://mlsanity.com/paper/2403.00815/) (2024)
- [Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarization](https://mlsanity.com/paper/2209.00930/) (2022)
- [Retrieval Augmentation for Commonsense Reasoning: A Unified Approach](https://mlsanity.com/paper/2210.12887/) (2022)
- [Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models](https://mlsanity.com/paper/2109.03892/) (2021)
- [SAPPHIRE: Approaches for Enhanced Concept-to-Text Generation](https://mlsanity.com/paper/2108.06643/) (2021)
- [Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning](https://mlsanity.com/paper/2210.11708/) (2022)
- [Expository Text Generation: Imitate, Retrieve, Paraphrase](https://mlsanity.com/paper/2305.03276/) (2023)
- [Contextualized Scene Imagination for Generative Commonsense Reasoning](https://mlsanity.com/paper/2112.06318/) (2021)
- [KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation](https://mlsanity.com/paper/2112.08266/) (2021)
- [Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach](https://mlsanity.com/paper/2205.13183/) (2022)
- [CoSe-Co: Text Conditioned Generative CommonSense Contextualizer](https://mlsanity.com/paper/2206.05706/) (2022)
- [VER: Unifying Verbalizing Entities and Relations](https://mlsanity.com/paper/2211.11093/) (2022)
- [Lexical Complexity Controlled Sentence Generation](https://mlsanity.com/paper/2211.14540/) (2022)
- [ConceptNet infused DialoGPT for Underlying Commonsense Understanding and Reasoning in Dialogue Response Generation](https://mlsanity.com/paper/2209.15109/) (2022)
- [LLM-R: A Framework for Domain-Adaptive Maintenance Scheme Generation Combining Hierarchical Agents and RAG](https://mlsanity.com/paper/2411.04476/) (2024)
- [Do LLMs exhibit the same commonsense capabilities across languages?](https://mlsanity.com/paper/2509.06401/) (2025)

## Abstract (arXiv)

Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsense knowledge and compositional generalization ability even puzzles strong pre-trained language generation models. We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. For fine-tuning, we further boost its performance with a trainable sentence retriever. We demonstrate experimentally on the large-scale CommonGen benchmark that our approach achieves new state-of-the-art results.