# Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge

arXiv: https://arxiv.org/abs/2104.05216 · HTML: https://mlsanity.com/paper/2104.05216/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [Revisiting Few-sample BERT Fine-tuning](https://mlsanity.com/paper/2006.05987/) (2020)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [A Survey on Network Embedding](https://mlsanity.com/paper/1711.08752/) (2017)
- [TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection](https://mlsanity.com/paper/1911.04118/) (2019)
- [Improved Neural Relation Detection for Knowledge Base Question Answering](https://mlsanity.com/paper/1704.06194/) (2017)
- [Leveraging Knowledge Bases in LSTMs for Improving Machine Reading](https://mlsanity.com/paper/1902.09091/) (2019)
- [ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs](https://mlsanity.com/paper/1512.05193/) (2015)
- [Attentive Pooling Networks](https://mlsanity.com/paper/1602.03609/) (2016)
- [Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answering](https://mlsanity.com/paper/1911.09801/) (2019)
- [Applying Deep Learning to Answer Selection: A Study and An Open Task](https://mlsanity.com/paper/1508.01585/) (2015)
- [Supervised and Unsupervised Transfer Learning for Question Answering](https://mlsanity.com/paper/1711.05345/) (2017)
- [Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering](https://mlsanity.com/paper/1707.07847/) (2017)
- [A Compare-Aggregate Model with Latent Clustering for Answer Selection](https://mlsanity.com/paper/1905.12897/) (2019)
- [Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture](https://mlsanity.com/paper/1707.06372/) (2017)
- [CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases](https://mlsanity.com/paper/1606.01994/) (2016)
- [Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering](https://mlsanity.com/paper/1812.02354/) (2018)
- [A Gated Self-attention Memory Network for Answer Selection](https://mlsanity.com/paper/1909.09696/) (2019)
- [Answer Ranking for Product-Related Questions via Multiple Semantic Relations Modeling](https://mlsanity.com/paper/2006.15599/) (2020)
- [BERTSel: Answer Selection with Pre-trained Models](https://mlsanity.com/paper/1905.07588/) (2019)
- [Review-guided Helpful Answer Identification in E-commerce](https://mlsanity.com/paper/2003.06209/) (2020)

## Built on by (most cited first, 7 of 7)

- [PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance](https://mlsanity.com/paper/2210.08817/) (2022)
- [Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations](https://mlsanity.com/paper/2402.15062/) (2024)
- [Towards Personalized Answer Generation in E-Commerce via Multi-Perspective Preference Modeling](https://mlsanity.com/paper/2112.13556/) (2021)
- [Question-Answer Sentence Graph for Joint Modeling Answer Selection](https://mlsanity.com/paper/2203.03549/) (2022)
- [ReliCD: A Reliable Cognitive Diagnosis Framework with Confidence Awareness](https://mlsanity.com/paper/2401.10749/) (2023)
- [DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text](https://mlsanity.com/paper/2310.12557/) (2023)
- [Learning to Rank Question Answer Pairs with Bilateral Contrastive Data Augmentation](https://mlsanity.com/paper/2106.11096/) (2021)

## Abstract (arXiv)

Answer selection, which is involved in many natural language processing applications such as dialog systems and question answering (QA), is an important yet challenging task in practice, since conventional methods typically suffer from the issues of ignoring diverse real-world background knowledge. In this paper, we extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG). First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network (KNN), which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. Then, we develop two kinds of knowledge-aware attention mechanism to summarize both the context-based and knowledge-based interactions between questions and answers. To handle the diversity and complexity of KG information, we further propose a Contextualized Knowledge-aware Attentive Neural Network (CKANN), which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network (GCN) and comprehensively learns context-based and knowledge-based sentence representation via the multi-view knowledge-aware attention mechanism. We evaluate our method on four widely-used benchmark QA datasets, including WikiQA, TREC QA, InsuranceQA and Yahoo QA. Results verify the benefits of incorporating external knowledge from KG, and show the robust superiority and extensive applicability of our method.