# Improving Question Answering by Commonsense-Based Pre-Training

arXiv: https://arxiv.org/abs/1809.03568 · HTML: https://mlsanity.com/paper/1809.03568/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2018.

## Built on by (most cited first, 30 of 66)

- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Knowledge Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2205.00976/) (2022)
- [Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2303.13948/) (2023)
- [A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation](https://mlsanity.com/paper/2104.13030/) (2021)
- [Graph Learning based Recommender Systems: A Review](https://mlsanity.com/paper/2105.06339/) (2021)
- [RevCore: Review-augmented Conversational Recommendation](https://mlsanity.com/paper/2106.00957/) (2021)
- [DiffKG: Knowledge Graph Diffusion Model for Recommendation](https://mlsanity.com/paper/2312.16890/) (2023)
- [Thermometer: Towards Universal Calibration for Large Language Models](https://mlsanity.com/paper/2403.08819/) (2024)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applications](https://mlsanity.com/paper/2206.07922/) (2022)
- [ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning](https://mlsanity.com/paper/2309.01538/) (2023)
- [Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2207.06236/) (2022)
- [VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge](https://mlsanity.com/paper/2210.13626/) (2022)
- [DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation](https://mlsanity.com/paper/2206.10848/) (2022)
- [Graph-Enhanced Multi-Task Learning of Multi-Level Transition Dynamics for Session-based Recommendation](https://mlsanity.com/paper/2110.03996/) (2021)
- [Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendations](https://mlsanity.com/paper/2311.10779/) (2023)
- [Semantic Structure Enhanced Event Causality Identification](https://mlsanity.com/paper/2305.12792/) (2023)
- [Reinforced Path Reasoning for Counterfactual Explainable Recommendation](https://mlsanity.com/paper/2207.06674/) (2022)
- [Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments](https://mlsanity.com/paper/2212.07425/) (2022)
- [LogiGAN: Learning Logical Reasoning via Adversarial Pre-training](https://mlsanity.com/paper/2205.08794/) (2022)
- [Conditional Attention Networks for Distilling Knowledge Graphs in Recommendation](https://mlsanity.com/paper/2111.02100/) (2021)
- [A Simple But Powerful Graph Encoder for Temporal Knowledge Graph Completion](https://mlsanity.com/paper/2112.07791/) (2021)
- [Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information](https://mlsanity.com/paper/2211.08169/) (2022)
- [Symmetric Graph Contrastive Learning against Noisy Views for Recommendation](https://mlsanity.com/paper/2408.02691/) (2024)
- [PR-Net: Preference Reasoning for Personalized Video Highlight Detection](https://mlsanity.com/paper/2109.01799/) (2021)
- [Learning Meta Representations of One-shot Relations for Temporal Knowledge Graph Link Prediction](https://mlsanity.com/paper/2205.10621/) (2022)
- [Aligning Recommendation and Conversation via Dual Imitation](https://mlsanity.com/paper/2211.02848/) (2022)
- [Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models](https://mlsanity.com/paper/2412.13544/) (2024)
- [Automatic Meta-Path Discovery for Effective Graph-Based Recommendation](https://mlsanity.com/paper/2112.12845/) (2021)

## Abstract (arXiv)

Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge. We believe the main reason is the lack of commonsense mboxconnections between concepts. To remedy this, we provide a simple and effective method that leverages external commonsense knowledge base such as ConceptNet. We pre-train direct and indirect relational functions between concepts, and show that these pre-trained functions could be easily added to existing neural network models. Results show that incorporating commonsense-based function improves the baseline on three question answering tasks that require commonsense reasoning. Further analysis shows that our system mboxdiscovers and leverages useful evidence from an external commonsense knowledge base, which is missing in existing neural network models and help derive the correct answer.