mlsanityData to 7 Oct 2026

SoftQE: Learned Representations of Queries Expanded by LLMs

arXiv 2402.12663 ↗First posted 20 Feb 2024cs.CL cs.IR cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We investigate the integration of Large Language Models (LLMs) into query encoders to improve dense retrieval without increasing latency and cost, by circumventing the dependency on LLMs at inference time. SoftQE incorporates knowledge from LLMs by mapping embeddings of input queries to those of the LLM-expanded queries. While improvements over various strong baselines on in-domain MS-MARCO metrics are marginal, SoftQE improves performance by 2.83 absolute percentage points on average on five out-of-domain BEIR tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

24 of 30 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  10. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  11. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  12. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
12 more
  1. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  4. 2020Overview of the TREC 2019 deep learning trackcited by 282
  5. 2021Overview of the TREC 2020 deep learning trackcited by 193
  6. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  7. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  8. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  9. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  10. 2021Learning Passage Impacts for Inverted Indexescited by 68
  11. 2023Query Expansion by Prompting Large Language Modelscited by 56
  12. 2023Doc2Query--: When Less is Morecited by 20

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Zero-Shot Dense Retrieval with Embeddings from Relevance Feedbackcited by 3