# Leveraging Knowledge in Multilingual Commonsense Reasoning

arXiv: https://arxiv.org/abs/2110.08462 · HTML: https://mlsanity.com/paper/2110.08462/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 16 Oct 2021.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference](https://mlsanity.com/paper/1808.05326/) (2018)
- [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://mlsanity.com/paper/2005.00333/) (2020)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning](https://mlsanity.com/paper/1909.02151/) (2019)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering](https://mlsanity.com/paper/2005.00646/) (2020)
- [A Survey of Knowledge-Enhanced Text Generation](https://mlsanity.com/paper/2010.04389/) (2020)
- [A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation](https://mlsanity.com/paper/2001.05139/) (2020)
- [Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning](https://mlsanity.com/paper/2106.09226/) (2021)
- [Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning](https://mlsanity.com/paper/2106.06937/) (2021)
- [Fusing Context Into Knowledge Graph for Commonsense Question Answering](https://mlsanity.com/paper/2012.04808/) (2020)
- [XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge](https://mlsanity.com/paper/2109.12573/) (2021)
- [Incorporating External Knowledge into Machine Reading for Generative Question Answering](https://mlsanity.com/paper/1909.02745/) (2019)
- [Does Knowledge Help General NLU? An Empirical Study](https://mlsanity.com/paper/2109.00563/) (2021)

## Built on by (most cited first, 9 of 9)

- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention](https://mlsanity.com/paper/2112.03254/) (2021)
- [Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints](https://mlsanity.com/paper/2406.18085/) (2024)
- [A Commonsense-Infused Language-Agnostic Learning Framework for Enhancing Prediction of Political Polarity in Multilingual News Headlines](https://mlsanity.com/paper/2212.00298/) (2022)
- [RTSUM: Relation Triple-based Interpretable Summarization with Multi-level Salience Visualization](https://mlsanity.com/paper/2310.13895/) (2023)
- [MindRef: Mimicking Human Memory for Hierarchical Reference Retrieval with Fine-Grained Location Awareness](https://mlsanity.com/paper/2402.17010/) (2024)
- [CASE: Commonsense-Augmented Score with an Expanded Answer Space](https://mlsanity.com/paper/2311.01684/) (2023)
- [Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models](https://mlsanity.com/paper/2410.13192/) (2024)

## Abstract (arXiv)

Commonsense reasoning (CSR) requires the model to be equipped with general world knowledge. While CSR is a language-agnostic process, most comprehensive knowledge sources are in few popular languages, especially English. Thus, it remains unclear how to effectively conduct multilingual commonsense reasoning (XCSR) for various languages. In this work, we propose to utilize English knowledge sources via a translate-retrieve-translate (TRT) strategy. For multilingual commonsense questions and choices, we collect related knowledge via translation and retrieval from the knowledge sources. The retrieved knowledge is then translated into the target language and integrated into a pre-trained multilingual language model via visible knowledge attention. Then we utilize a diverse of 4 English knowledge sources to provide more comprehensive coverage of knowledge in different formats. Extensive results on the XCSR benchmark demonstrate that TRT with external knowledge can significantly improve multilingual commonsense reasoning in both zero-shot and translate-train settings, outperforming 3.3 and 3.6 points over the previous state-of-the-art on XCSR benchmark datasets (X-CSQA and X-CODAH).