Decomposing and Revising What Language Models Generate
Not on the map. Why?
Abstract (arXiv)
Attribution is crucial in question answering (QA) with Large Language Models (LLMs).SOTA question decomposition-based approaches use long form answers to generate questions for retrieving related documents. However, the generated questions are often irrelevant and incomplete, resulting in a loss of facts in retrieval.These approaches also fail to aggregate evidence snippets from different documents and paragraphs. To tackle these problems, we propose a new fact decomposition-based framework called FIDES (textitfaithful context enhanced fact decomposition and evidence aggregation) for attributed QA. FIDES uses a contextually enhanced two-stage faithful decomposition method to decompose long form answers into sub-facts, which are then used by a retriever to retrieve related evidence snippets. If the retrieved evidence snippets conflict with the related sub-facts, such sub-facts will be revised accordingly. Finally, the evidence snippets are aggregated according to the original sentences.Extensive evaluation has been conducted with six datasets, with an additionally proposed new metric called Attrauto-P for evaluating the evidence precision. FIDES outperforms the SOTA methods by over 14% in average with GPT-3.5-turbo, Gemini and Llama 70B series.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 83 of its 146 references that are in the map. First posted on arXiv 31 Aug 2025.
Builds on
83 of 146 references.
References · 83
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
38 more
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
- 2023Enabling Large Language Models to Generate Text with Citationscited by 595
- 2015Holistically-Nested Edge Detectioncited by 592
- 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
- 2021Intriguing Properties of Vision Transformerscited by 446
- 2018The UCR Time Series Archivecited by 431
- 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 410
- 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
- 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
- 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
- 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
- 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
- 2021Measuring Attribution in Natural Language Generation Modelscited by 256
- 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
- 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
- 2020Representation Learning via Invariant Causal Mechanismscited by 199
- 2021Partial success in closing the gap between human and machine visioncited by 185
- 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
- 2016Exploring the Limits of Language Modelingcited by 173
- 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 171
- 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
- 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 144
- 2021Decoupled Contrastive Learningcited by 143
- 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 115
- 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 112
50 most cited of 83 All 83 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.