# Retrieval augmentation of large language models for lay language generation

arXiv: https://arxiv.org/abs/2211.03818 · HTML: https://mlsanity.com/paper/2211.03818/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 79 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Builds on (most cited first, 30 of 37)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports](https://mlsanity.com/paper/1911.02541/) (2019)
- [Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization](https://mlsanity.com/paper/2110.07356/) (2021)
- [Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature](https://mlsanity.com/paper/2210.09932/) (2022)
- [Learning to Summarize Radiology Findings](https://mlsanity.com/paper/1809.04698/) (2018)
- [Paragraph-level Simplification of Medical Texts](https://mlsanity.com/paper/2104.05767/) (2021)
- [Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization](https://mlsanity.com/paper/2008.11293/) (2020)
- [Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures](https://mlsanity.com/paper/2009.08666/) (2020)
- [Automated Lay Language Summarization of Biomedical Scientific Reviews](https://mlsanity.com/paper/2012.12573/) (2020)
- [Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen](https://mlsanity.com/paper/2005.00701/) (2020)
- [Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations](https://mlsanity.com/paper/2109.12174/) (2021)
- [Evaluating Factuality in Text Simplification](https://mlsanity.com/paper/2204.07562/) (2022)
- [Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification](https://mlsanity.com/paper/2010.10035/) (2020)

## Built on by (most cited first, 21 of 21)

- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP](https://mlsanity.com/paper/2312.15561/) (2023)
- [APPLS: Evaluating Evaluation Metrics for Plain Language Summarization](https://mlsanity.com/paper/2305.14341/) (2023)
- [Multilingual Simplification of Medical Texts](https://mlsanity.com/paper/2305.12532/) (2023)
- [A Fine-tuning Enhanced RAG System with Quantized Influence Measure as AI Judge](https://mlsanity.com/paper/2402.17081/) (2024)
- [Clinical Insights: A Comprehensive Review of Language Models in Medicine](https://mlsanity.com/paper/2408.11735/) (2024)
- [Evaluating the Evaluators: Are readability metrics good measures of readability?](https://mlsanity.com/paper/2508.19221/) (2025)
- [A LongFormer-Based Framework for Accurate and Efficient Medical Text Summarization](https://mlsanity.com/paper/2503.06888/) (2025)
- [Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation](https://mlsanity.com/paper/2508.08730/) (2025)
- [PlainQAFact: Retrieval-augmented Factual Consistency Evaluation Metric for Biomedical Plain Language Summarization](https://mlsanity.com/paper/2503.08890/) (2025)
- [LLM-based Text Simplification and its Effect on User Comprehension and Cognitive Load](https://mlsanity.com/paper/2505.01980/) (2025)
- [Addressing Hallucinations with RAG and NMISS in Italian Healthcare LLM Chatbots](https://mlsanity.com/paper/2412.04235/) (2024)
- [Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing](https://mlsanity.com/paper/2505.18867/) (2025)
- [Enhancing Large Language Models with Domain-specific Retrieval Augment Generation: A Case Study on Long-form Consumer Health Question Answering in Ophthalmology](https://mlsanity.com/paper/2409.13902/) (2024)
- [Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation](https://mlsanity.com/paper/2505.10409/) (2025)
- [Background Knowledge Grounding for Readable, Relevant, and Factual Biomedical Lay Summaries](https://mlsanity.com/paper/2305.02104/) (2023)
- [Stochastic Parrots or ICU Experts? Large Language Models in Critical Care Medicine: A Scoping Review](https://mlsanity.com/paper/2407.19256/) (2024)
- [Large language models management of medications: three performance analyses](https://mlsanity.com/paper/2509.22926/) (2025)
- [Not too long do read: Evaluating LLM-generated extreme scientific summaries](https://mlsanity.com/paper/2512.23206/) (2025)
- [Improving Health Literacy through Lay Summarization of Radiological Reports: An Evaluation of BioNER and Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.02396/) (2026)
- [Hierarchical Wasserstein Merging for Multi-Domain Multi-Task Learning: From Specialists to a Generalist](https://mlsanity.com/paper/2609.06406/) (2026)

## Abstract (arXiv)

Recent lay language generation systems have used Transformer models trained on a parallel corpus to increase health information accessibility. However, the applicability of these models is constrained by the limited size and topical breadth of available corpora. We introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. The abstract and the corresponding lay language summary are written by domain experts, assuring the quality of our dataset. Furthermore, qualitative evaluation of expert-authored plain language summaries has revealed background explanation as a key strategy to increase accessibility. Such explanation is challenging for neural models to generate because it goes beyond simplification by adding content absent from the source. We derive two specialized paired corpora from CELLS to address key challenges in lay language generation: generating background explanations and simplifying the original abstract. We adopt retrieval-augmented models as an intuitive fit for the task of background explanation generation, and show improvements in summary quality and simplicity while maintaining factual correctness. Taken together, this work presents the first comprehensive study of background explanation for lay language generation, paving the path for disseminating scientific knowledge to a broader audience. CELLS is publicly available at: https://github.com/LinguisticAnomalies/pls_retrieval.