# QASE Enhanced PLMs: Improved Control in Text Generation for MRC

arXiv: https://arxiv.org/abs/2403.04771 · HTML: https://mlsanity.com/paper/2403.04771/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering](https://mlsanity.com/paper/2104.08202/) (2021)
- [Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning](https://mlsanity.com/paper/1908.05803/) (2019)
- [Retrieve and Refine: Improved Sequence Generation Models For Dialogue](https://mlsanity.com/paper/1808.04776/) (2018)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [A Controllable Model of Grounded Response Generation](https://mlsanity.com/paper/2005.00613/) (2020)
- [Read before Generate! Faithful Long Form Question Answering with Machine Reading](https://mlsanity.com/paper/2203.00343/) (2022)
- [A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning](https://mlsanity.com/paper/1908.05514/) (2019)
- [Question Directed Graph Attention Network for Numerical Reasoning over Text](https://mlsanity.com/paper/2009.07448/) (2020)
- [Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering](https://mlsanity.com/paper/1811.11934/) (2018)
- [RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation](https://mlsanity.com/paper/2306.07206/) (2023)
- [Prototype-to-Style: Dialogue Generation with Style-Aware Editing on Retrieval Memory](https://mlsanity.com/paper/2004.02214/) (2020)
- [LIQUID: A Framework for List Question Answering Dataset Generation](https://mlsanity.com/paper/2302.01691/) (2023)
- [A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension](https://mlsanity.com/paper/1905.12848/) (2019)
- [Transductive Learning for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2109.07812/) (2021)
- [A Deep Cascade Model for Multi-Document Reading Comprehension](https://mlsanity.com/paper/1811.11374/) (2018)
- [Multi-span Style Extraction for Generative Reading Comprehension](https://mlsanity.com/paper/2009.07382/) (2020)
- [ArgU: A Controllable Factual Argument Generator](https://mlsanity.com/paper/2305.05334/) (2023)
- [INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2306.06381/) (2023)
- [How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension](https://mlsanity.com/paper/2306.00435/) (2023)

## Abstract (arXiv)

To address the challenges of out-of-control generation in generative models for machine reading comprehension (MRC), we introduce the Question-Attended Span Extraction (QASE) module. Integrated during the fine-tuning of pre-trained generative language models (PLMs), QASE enables these PLMs to match SOTA extractive methods and outperform leading LLMs like GPT-4 in MRC tasks, without significant increases in computational costs.