# Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction

arXiv: https://arxiv.org/abs/1806.00778 · HTML: https://mlsanity.com/paper/1806.00778/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2018.

## Built on by (most cited first, 4 of 4)

- [Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering](https://mlsanity.com/paper/2109.07009/) (2021)
- [DCAP: Deep Cross Attentional Product Network for User Response Prediction](https://mlsanity.com/paper/2105.08649/) (2021)
- [Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems](https://mlsanity.com/paper/2201.05767/) (2022)
- [Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision](https://mlsanity.com/paper/2209.15301/) (2022)

## Abstract (arXiv)

Attention is typically used to select informative sub-phrases that are used for prediction. This paper investigates the novel use of attention as a form of feature augmentation, i.e, casted attention. We propose Multi-Cast Attention Networks (MCAN), a new attention mechanism and general model architecture for a potpourri of ranking tasks in the conversational modeling and question answering domains. Our approach performs a series of soft attention operations, each time casting a scalar feature upon the inner word embeddings. The key idea is to provide a real-valued hint (feature) to a subsequent encoder layer and is targeted at improving the representation learning process. There are several advantages to this design, e.g., it allows an arbitrary number of attention mechanisms to be casted, allowing for multiple attention types (e.g., co-attention, intra-attention) and attention variants (e.g., alignment-pooling, max-pooling, mean-pooling) to be executed simultaneously. This not only eliminates the costly need to tune the nature of the co-attention layer, but also provides greater extents of explainability to practitioners. Via extensive experiments on four well-known benchmark datasets, we show that MCAN achieves state-of-the-art performance. On the Ubuntu Dialogue Corpus, MCAN outperforms existing state-of-the-art models by 9%. MCAN also achieves the best performing score to date on the well-studied TrecQA dataset.