# LoGU: Long-form Generation with Uncertainty Expressions

arXiv: https://arxiv.org/abs/2410.14309 · HTML: https://mlsanity.com/paper/2410.14309/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification](https://mlsanity.com/paper/2403.04696/) (2024)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales](https://mlsanity.com/paper/2405.20974/) (2024)
- [Unfamiliar Finetuning Examples Control How Language Models Hallucinate](https://mlsanity.com/paper/2403.05612/) (2024)
- [LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K](https://mlsanity.com/paper/2402.05136/) (2024)
- [LUQ: Long-text Uncertainty Quantification for LLMs](https://mlsanity.com/paper/2403.20279/) (2024)
- [Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation](https://mlsanity.com/paper/2402.09267/) (2024)
- [Linguistic Calibration of Long-Form Generations](https://mlsanity.com/paper/2404.00474/) (2024)
- [WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries](https://mlsanity.com/paper/2407.17468/) (2024)
- [Atomic Calibration of LLMs in Long-Form Generations](https://mlsanity.com/paper/2410.13246/) (2024)
- [Can Large Language Models Generate High-quality Patent Claims?](https://mlsanity.com/paper/2406.19465/) (2024)
- [Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning](https://mlsanity.com/paper/2406.10099/) (2024)

## Built on by (most cited first, 19 of 19)

- [Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results](https://mlsanity.com/paper/2504.13677/) (2025)
- [Demystifying Multi-Agent Debate: The Role of Confidence and Diversity](https://mlsanity.com/paper/2601.19921/) (2026)
- [LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations](https://mlsanity.com/paper/2505.23912/) (2025)
- [Shifting Long-Context LLMs Research from Input to Output](https://mlsanity.com/paper/2503.04723/) (2025)
- [Confidence Estimation for LLMs in Multi-turn Interactions](https://mlsanity.com/paper/2601.02179/) (2026)
- [UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation](https://mlsanity.com/paper/2505.16922/) (2025)
- [Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models](https://mlsanity.com/paper/2510.03136/) (2025)
- [SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?](https://mlsanity.com/paper/2505.20295/) (2025)
- [Possible or Definite? A Benchmark for Evaluating Diagnostic Uncertainty Preservation in Clinical Text](https://mlsanity.com/paper/2606.18471/) (2026)
- [Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning](https://mlsanity.com/paper/2502.11962/) (2025)
- [Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction](https://mlsanity.com/paper/2504.17671/) (2025)
- [GrACE: A Generative Approach to Better Confidence Elicitation and Efficient Test-Time Scaling in Large Language Models](https://mlsanity.com/paper/2509.09438/) (2025)
- [All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning](https://mlsanity.com/paper/2509.12908/) (2025)
- [LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability](https://mlsanity.com/paper/2510.24345/) (2025)
- [Towards Generation-Efficient Uncertainty Estimation in Large Language Models](https://mlsanity.com/paper/2605.06053/) (2026)
- [When Calibration Rankings Reverse: Accuracy-Controlled Evaluation for Fair Comparison of LLMs](https://mlsanity.com/paper/2606.30814/) (2026)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation](https://mlsanity.com/paper/2510.21891/) (2025)
- [Beyond Semantic Equivalence: Logical Graphs for LLM Uncertainty Quantification](https://mlsanity.com/paper/2607.16868/) (2026)

## Abstract (arXiv)

While Large Language Models (LLMs) demonstrate impressive capabilities, they still struggle with generating factually incorrect content (i.e., hallucinations). A promising approach to mitigate this issue is enabling models to express uncertainty when unsure. Previous research on uncertainty modeling has primarily focused on short-form QA, but realworld applications often require much longer responses. In this work, we introduce the task of Long-form Generation with Uncertainty(LoGU). We identify two key challenges: Uncertainty Suppression, where models hesitate to express uncertainty, and Uncertainty Misalignment, where models convey uncertainty inaccurately. To tackle these challenges, we propose a refinement-based data collection framework and a two-stage training pipeline. Our framework adopts a divide-and-conquer strategy, refining uncertainty based on atomic claims. The collected data are then used in training through supervised fine-tuning (SFT) and direct preference optimization (DPO) to enhance uncertainty expression. Extensive experiments on three long-form instruction following datasets show that our method significantly improves accuracy, reduces hallucinations, and maintains the comprehensiveness of responses.