# Visconde: Multi-document QA with GPT-3 and Neural Reranking

arXiv: https://arxiv.org/abs/2212.09656 · HTML: https://mlsanity.com/paper/2212.09656/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 37 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval](https://mlsanity.com/paper/2009.12756/) (2020)
- [Internet-augmented language models through few-shot prompting for open-domain question answering](https://mlsanity.com/paper/2203.05115/) (2022)
- [Faithful Reasoning Using Large Language Models](https://mlsanity.com/paper/2208.14271/) (2022)
- [Unsupervised Question Decomposition for Question Answering](https://mlsanity.com/paper/2002.09758/) (2020)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [IIRC: A Dataset of Incomplete Information Reading Comprehension Questions](https://mlsanity.com/paper/2011.07127/) (2020)
- [Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering](https://mlsanity.com/paper/1905.05733/) (2019)
- [Questions Are All You Need to Train a Dense Passage Retriever](https://mlsanity.com/paper/2206.10658/) (2022)
- [Multi-Hop Paragraph Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/1906.06606/) (2019)
- [Adapting Pretrained Text-to-Text Models for Long Text Sequences](https://mlsanity.com/paper/2209.10052/) (2022)
- [Boosting Search Engines with Interactive Agents](https://mlsanity.com/paper/2109.00527/) (2021)

## Built on by (most cited first, 23 of 23)

- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [Knowledge Graph Prompting for Multi-Document Question Answering](https://mlsanity.com/paper/2308.11730/) (2023)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [The Chronicles of RAG: The Retriever, the Chunk and the Generator](https://mlsanity.com/paper/2401.07883/) (2024)
- [PDFTriage: Question Answering over Long, Structured Documents](https://mlsanity.com/paper/2309.08872/) (2023)
- [Characterizing LLM Abstention Behavior in Science QA with Context Perturbations](https://mlsanity.com/paper/2404.12452/) (2024)
- [DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering](https://mlsanity.com/paper/2406.07348/) (2024)
- [Drilling Down into the Discourse Structure with LLMs for Long Document Question Answering](https://mlsanity.com/paper/2311.13565/) (2023)
- [Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models](https://mlsanity.com/paper/2309.08922/) (2023)
- [FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipeline](https://mlsanity.com/paper/2410.13959/) (2024)
- [Enhancing In-Context Learning with Answer Feedback for Multi-Span Question Answering](https://mlsanity.com/paper/2306.04508/) (2023)
- [POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval](https://mlsanity.com/paper/2505.19189/) (2025)
- [LitSumm: Large language models for literature summarisation of non-coding RNAs](https://mlsanity.com/paper/2311.03056/) (2023)
- [Towards Self-Contained Answers: Entity-Based Answer Rewriting in Conversational Search](https://mlsanity.com/paper/2403.01747/) (2024)
- [Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts](https://mlsanity.com/paper/2404.02022/) (2024)
- [NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering](https://mlsanity.com/paper/2505.19754/) (2025)
- [Leveraging Inter-Chunk Interactions for Enhanced Retrieval in Large Language Model-Based Question Answering](https://mlsanity.com/paper/2408.02907/) (2024)
- [EviMem: Evidence-Gap-Driven Iterative Retrieval for Long-Term Conversational Memory](https://mlsanity.com/paper/2604.27695/) (2026)
- [MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answering](https://mlsanity.com/paper/2310.06436/) (2023)
- [Explaining Mixtures of Sources in News Articles](https://mlsanity.com/paper/2411.05192/) (2024)
- [Evaluation of Faithfulness Using the Longest Supported Subsequence](https://mlsanity.com/paper/2308.12157/) (2023)
- [INACIA: Integrating Large Language Models in Brazilian Audit Courts: Opportunities and Challenges](https://mlsanity.com/paper/2401.05273/) (2024)
- [Gradually Excavating External Knowledge for Implicit Complex Question Answering](https://mlsanity.com/paper/2603.08148/) (2026)

## Abstract (arXiv)

This paper proposes a question-answering system that can answer questions whose supporting evidence is spread over multiple (potentially long) documents. The system, called Visconde, uses a three-step pipeline to perform the task: decompose, retrieve, and aggregate. The first step decomposes the question into simpler questions using a few-shot large language model (LLM). Then, a state-of-the-art search engine is used to retrieve candidate passages from a large collection for each decomposed question. In the final step, we use the LLM in a few-shot setting to aggregate the contents of the passages into the final answer. The system is evaluated on three datasets: IIRC, Qasper, and StrategyQA. Results suggest that current retrievers are the main bottleneck and that readers are already performing at the human level as long as relevant passages are provided. The system is also shown to be more effective when the model is induced to give explanations before answering a question. Code is available at urlhttps://github.com/neuralmind-ai/visconde.