# Query Refinement Prompts for Closed-Book Long-Form Question Answering

arXiv: https://arxiv.org/abs/2210.17525 · HTML: https://mlsanity.com/paper/2210.17525/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Learning by Distilling Context](https://mlsanity.com/paper/2209.15189/) (2022)
- [Conditional Generation with a Question-Answering Blueprint](https://mlsanity.com/paper/2207.00397/) (2022)
- [AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization](https://mlsanity.com/paper/2010.12694/) (2020)
- [Quizbowl: The Case for Incremental Question Answering](https://mlsanity.com/paper/1904.04792/) (2019)
- [Control Prefixes for Parameter-Efficient Text Generation](https://mlsanity.com/paper/2110.08329/) (2021)
- [PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization](https://mlsanity.com/paper/2204.04413/) (2022)

## Built on by (most cited first, 12 of 12)

- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2310.14696/) (2023)
- [Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agents](https://mlsanity.com/paper/2402.17896/) (2024)
- [FoRAG: Factuality-optimized Retrieval Augmented Generation for Web-enhanced Long-form Question Answering](https://mlsanity.com/paper/2406.13779/) (2024)
- [Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering](https://mlsanity.com/paper/2409.02361/) (2024)
- [EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes](https://mlsanity.com/paper/2509.00877/) (2025)
- [Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey](https://mlsanity.com/paper/2505.12543/) (2025)
- [PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering](https://mlsanity.com/paper/2310.16147/) (2023)
- [A Benchmark Dataset with Larger Context for Non-Factoid Question Answering over Islamic Text](https://mlsanity.com/paper/2409.09844/) (2024)
- [Evaluation of medium-large Language Models at zero-shot closed book generative question answering](https://mlsanity.com/paper/2305.11991/) (2023)
- [Retrieving Contextual Information for Long-Form Question Answering using Weak Supervision](https://mlsanity.com/paper/2410.08623/) (2024)
- [Diversity is Not Ambiguity: Toward Accurate and Efficient Ambiguity Detection for Open-Domain QA](https://mlsanity.com/paper/2608.03177/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings. While the former can be validated using well-known evaluation metrics, the latter is difficult to evaluate. We resolve the difficulties to evaluate long-form output by doing both tasks at once -- to do question answering that requires long-form answers. Such questions tend to be multifaceted, i.e., they may have ambiguities and/or require information from multiple sources. To this end, we define query refinement prompts that encourage LLMs to explicitly express the multifacetedness in questions and generate long-form answers covering multiple facets of the question. Our experiments on two long-form question answering datasets, ASQA and AQuAMuSe, show that using our prompts allows us to outperform fully finetuned models in the closed book setting, as well as achieve results comparable to retrieve-then-generate open-book models.