# Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection

arXiv: https://arxiv.org/abs/2011.07208 · HTML: https://mlsanity.com/paper/2011.07208/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarization](https://mlsanity.com/paper/2112.11670/) (2021)
- [Improving Named Entity Recognition in Telephone Conversations via Effective Active Learning with Human in the Loop](https://mlsanity.com/paper/2211.01354/) (2022)
- [CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning](https://mlsanity.com/paper/2404.09077/) (2024)
- [Extending Isolation Forest for Anomaly Detection in Big Data via K-Means](https://mlsanity.com/paper/2104.13190/) (2021)
- [SensPick: Sense Picking for Word Sense Disambiguation](https://mlsanity.com/paper/2102.05260/) (2021)
- [To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval](https://mlsanity.com/paper/2601.17500/) (2026)

## Abstract (arXiv)

Pre-training a transformer-based model for the language modeling task in a large dataset and then fine-tuning it for downstream tasks has been found very useful in recent years. One major advantage of such pre-trained language models is that they can effectively absorb the context of each word in a sentence. However, for tasks such as the answer selection task, the pre-trained language models have not been extensively used yet. To investigate their effectiveness in such tasks, in this paper, we adopt the pre-trained Bidirectional Encoder Representations from Transformer (BERT) language model and fine-tune it on two Question Answering (QA) datasets and three Community Question Answering (CQA) datasets for the answer selection task. We find that fine-tuning the BERT model for the answer selection task is very effective and observe a maximum improvement of 13.1% in the QA datasets and 18.7% in the CQA datasets compared to the previous state-of-the-art.