# ISQA: Informative Factuality Feedback for Scientific Summarization

arXiv: https://arxiv.org/abs/2404.13246 · HTML: https://mlsanity.com/paper/2404.13246/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 20 Apr 2024.

## Builds on (most cited first, 24 of 24)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [Learning From Mistakes Makes LLM Better Reasoner](https://mlsanity.com/paper/2310.20689/) (2023)
- [LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization](https://mlsanity.com/paper/2301.13298/) (2023)
- [Answers Unite! Unsupervised Metrics for Reinforced Summarization Models](https://mlsanity.com/paper/1909.01610/) (2019)
- [Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback](https://mlsanity.com/paper/2306.00186/) (2023)
- [CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs](https://mlsanity.com/paper/2112.08804/) (2021)
- [Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling](https://mlsanity.com/paper/2210.12378/) (2022)
- [On Improving Summarization Factual Consistency from Natural Language Feedback](https://mlsanity.com/paper/2212.09968/) (2022)
- [Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization](https://mlsanity.com/paper/2208.09770/) (2022)
- [UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization](https://mlsanity.com/paper/2211.09783/) (2022)
- [A Practical Toolkit for Multilingual Question and Answer Generation](https://mlsanity.com/paper/2305.17416/) (2023)

## Built on by (most cited first, 2 of 2)

- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [ScholarSum: Student-Teacher Abstractive Summarization via Knowledge Graph Reasoning and Reflective Refinement](https://mlsanity.com/paper/2606.18850/) (2026)

## Abstract (arXiv)

We propose Iterative Facuality Refining on Informative Scientific Question-Answering (ISQA) feedbackfootnoteCode is available at urlhttps://github.com/lizekai-richard/isqa, a method following human learning theories that employs model-generated feedback consisting of both positive and negative information. Through iterative refining of summaries, it probes for the underlying rationale of statements to enhance the factuality of scientific summarization. ISQA does this in a fine-grained manner by asking a summarization agent to reinforce validated statements in positive feedback and fix incorrect ones in negative feedback. Our findings demonstrate that the ISQA feedback mechanism significantly improves the factuality of various open-source LLMs on the summarization task, as evaluated across multiple scientific datasets.