# Sequencing Matters: A Generate-Retrieve-Generate Model for Building Conversational Agents

arXiv: https://arxiv.org/abs/2311.09513 · HTML: https://mlsanity.com/paper/2311.09513/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 21 of 21)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023)
- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [A Theoretical Analysis of NDCG Type Ranking Measures](https://mlsanity.com/paper/1304.6480/) (2013)
- [Yes, BM25 is a Strong Baseline for Legal Case Retrieval](https://mlsanity.com/paper/2105.05686/) (2021)
- [Adapting Pretrained Text-to-Text Models for Long Text Sequences](https://mlsanity.com/paper/2209.10052/) (2022)
- [Retrieving Supporting Evidence for LLMs Generated Answers](https://mlsanity.com/paper/2306.13781/) (2023)
- [ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information](https://mlsanity.com/paper/2211.15848/) (2022)
- [ChatGPT Hallucinates when Attributing Answers](https://mlsanity.com/paper/2309.09401/) (2023)
- [Context-Aware Answer Extraction in Question Answering](https://mlsanity.com/paper/2011.02687/) (2020)
- [Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation](https://mlsanity.com/paper/1902.08850/) (2019)
- [Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval](https://mlsanity.com/paper/2308.08285/) (2023)
- [An Unsupervised Semantic Sentence Ranking Scheme for Text Documents](https://mlsanity.com/paper/2005.02158/) (2020)
- [Fake News Detection with Different Models](https://mlsanity.com/paper/2003.04978/) (2020)
- [Adapting Sentence Transformers for the Aviation Domain](https://mlsanity.com/paper/2305.09556/) (2023)

## Built on by (most cited first, 1 of 1)

- [Adaptive Personalized Conversational Information Retrieval](https://mlsanity.com/paper/2508.08634/) (2025)

## Abstract (arXiv)

This paper contains what the Georgetown InfoSense group has done in regard to solving the challenges presented by TREC iKAT 2023. Our submitted runs outperform the median runs by a significant margin, exhibiting superior performance in nDCG across various cut numbers and in overall success rate. Our approach uses a Generate-Retrieve-Generate method, which we've found to greatly outpace Retrieve-Then-Generate approaches for the purposes of iKAT. Our solution involves the use of Large Language Models (LLMs) for initial answers, answer grounding by BM25, passage quality filtering by logistic regression, and answer generation by LLMs again. We leverage several purpose-built Language Models, including BERT, Chat-based, and text-to-transfer-based models, for text understanding, classification, generation, and summarization. The official results of the TREC evaluation contradict our initial self-evaluation, which may suggest that a decrease in the reliance on our retrieval and classification methods is better. Nonetheless, our findings suggest that the sequence of involving these different components matters, where we see an essentiality of using LLMs before using search engines.