mlsanityData to 14 Oct 2026

Thrust: Adaptively Propels Large Language Models with External Knowledge

arXiv 2307.10442 ↗First posted 19 Jul 2023cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Although large-scale pre-trained language models (PTLMs) are shown to encode rich knowledge in their model parameters, the inherent knowledge in PTLMs can be opaque or static, making external knowledge necessary. However, the existing information retrieval techniques could be costly and may even introduce noisy and sometimes misleading knowledge. To address these challenges, we propose the instance-level adaptive propulsion of external knowledge (IAPEK), where we only conduct the retrieval when necessary. To achieve this goal, we propose measuring whether a PTLM contains enough knowledge to solve an instance with a novel metric, Thrust, which leverages the representation distribution of a small number of seen instances. Extensive experiments demonstrate that thrust is a good measurement of PTLM models' instance-level knowledgeability. Moreover, we can achieve significantly higher cost-efficiency with the Thrust score as the retrieval indicator than the naive usage of external knowledge on 88% of the evaluated tasks with 26% average performance improvement. Such findings shed light on the real-world practice of knowledge-enhanced LMs with a limited knowledge-seeking budget due to computation latency or costs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 19 Jul 2023.

Builds on

42 of 56 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
30 more
  1. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  2. 2019Language Models as Knowledge Bases?cited by 1,936
  3. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  10. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  12. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  13. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  14. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  15. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  16. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  17. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  18. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  19. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  20. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  21. 2021LIME: Learning Inductive Bias for Primitives of Mathematical Reasoningcited by 46
  22. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  23. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  24. 2019A Surprisingly Robust Trick for Winograd Schema Challengecited by 30
  25. 2022A Survey of Knowledge-Intensive NLP with Pre-Trained Language Modelscited by 30
  26. 2021Sensitivity as a Complexity Measure for Sequence Classification Taskscited by 28
  27. 2022Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Modelscited by 24
  28. 2021Rissanen Data Analysis: Examining Dataset Characteristics via Description Lengthcited by 22
  29. 2020Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Datacited by 17
  30. 2022CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigmcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  2. 2025Improving Large Language Model Planning with Action Sequence Similaritycited by 7
  3. 2025Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filteringcited by 2