Speeding Up Question Answering Task of Language Models via Inverted Index
Not on the map. Why?
Abstract (arXiv)
Natural language processing applications, such as conversational agents and their question-answering capabilities, are widely used in the real world. Despite the wide popularity of large language models (LLMs), few real-world conversational agents take advantage of LLMs. Extensive resources consumed by LLMs disable developers from integrating them into end-user applications. In this study, we leverage an inverted indexing mechanism combined with LLMs to improve the efficiency of question-answering models for closed-domain questions. Our experiments show that using the index improves the average response time by 97.44%. In addition, due to the reduced search scope, the average BLEU score improved by 0.23 while using the inverted index.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 19 references that are in the map. First posted on arXiv 24 Oct 2022.
Builds on
6 of 19 references.
References · 6
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
- 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
- 2022ODSearch: Fast and Resource Efficient On-device Natural Language Search for Fitness Trackers' Datacited by 5
- 2022Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understandingcited by 2
Built on by
Not cited yet.