# Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers

arXiv: https://arxiv.org/abs/2401.04695 · HTML: https://mlsanity.com/paper/2401.04695/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Builds on (most cited first, 23 of 23)

- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Reframing Instructional Prompts to GPTk's Language](https://mlsanity.com/paper/2109.07830/) (2021)
- [The Capacity for Moral Self-Correction in Large Language Models](https://mlsanity.com/paper/2302.07459/) (2023)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)
- [R-Tuning: Instructing Large Language Models to Say `I Don't Know'](https://mlsanity.com/paper/2311.09677/) (2023)
- [Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation](https://mlsanity.com/paper/2305.09860/) (2023)
- [It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk](https://mlsanity.com/paper/2310.01387/) (2023)
- [Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs](https://mlsanity.com/paper/2310.11689/) (2023)
- [What's in a Name? Answer Equivalence For Open-Domain Question Answering](https://mlsanity.com/paper/2109.05289/) (2021)
- [Uncertainty-aware Language Modeling for Selective Question Answering](https://mlsanity.com/paper/2311.15451/) (2023)
- [Can Language Models Be Specific? How?](https://mlsanity.com/paper/2210.05159/) (2022)
- [Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment](https://mlsanity.com/paper/2307.03319/) (2023)

## Built on by (most cited first, 18 of 18)

- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [To Believe or Not to Believe Your LLM](https://mlsanity.com/paper/2406.02543/) (2024)
- [Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?](https://mlsanity.com/paper/2405.16908/) (2024)
- [A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice](https://mlsanity.com/paper/2410.15326/) (2024)
- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [Estimating Knowledge in Large Language Models Without Generating a Single Token](https://mlsanity.com/paper/2406.12673/) (2024)
- [Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness](https://mlsanity.com/paper/2510.09033/) (2025)
- [Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships](https://mlsanity.com/paper/2407.12543/) (2024)
- [Keep Guessing? When Considering Inference Scaling, Mind the Baselines](https://mlsanity.com/paper/2410.15466/) (2024)
- [InFact: Informativeness Alignment for Improved LLM Factuality](https://mlsanity.com/paper/2505.20487/) (2025)
- [When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation](https://mlsanity.com/paper/2602.11908/) (2026)
- [Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering](https://mlsanity.com/paper/2605.26620/) (2026)
- [Pretrained LLMs Learn Multiple Types of Uncertainty](https://mlsanity.com/paper/2505.21218/) (2025)
- [Inter-Passage Verification for Multi-evidence Multi-answer QA](https://mlsanity.com/paper/2506.00425/) (2025)
- [How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generation](https://mlsanity.com/paper/2606.12789/) (2026)
- [A Bi-consolidating Model for Joint Relational Triple Extraction](https://mlsanity.com/paper/2404.03881/) (2024)
- [HaS: Accelerating RAG through Homology-Aware Speculative Retrieval](https://mlsanity.com/paper/2604.20452/) (2026)
- [How Correct Is Your Answer? A Semantic Correctness Framework for Open QA Evaluation](https://mlsanity.com/paper/2609.01369/) (2026)

## Abstract (arXiv)

Factual questions typically can be answered correctly at different levels of granularity. For example, both ``August 4, 1961'' and ``1961'' are correct answers to the question ``When was Barack Obama born?''. Standard question answering (QA) evaluation protocols, however, do not explicitly take this into account and compare a predicted answer against answers of a single granularity level. In this work, we propose GRANOLA QA, a novel evaluation setting where a predicted answer is evaluated in terms of accuracy and informativeness against a set of multi-granularity answers. We present a simple methodology for enriching existing datasets with multi-granularity answers, and create GRANOLA-EQ, a multi-granularity version of the EntityQuestions dataset. We evaluate a range of decoding methods on GRANOLA-EQ, including a new algorithm, called Decoding with Response Aggregation (DRAG), that is geared towards aligning the response granularity with the model's uncertainty. Our experiments show that large language models with standard decoding tend to generate specific answers, which are often incorrect. In contrast, when evaluated on multi-granularity answers, DRAG yields a nearly 20 point increase in accuracy on average, which further increases for rare entities. Overall, this reveals that standard evaluation and decoding schemes may significantly underestimate the knowledge encapsulated in LMs.