Simplifying Scholarly Abstracts for Accessible Digital Libraries
Not on the map. Why?
Abstract (arXiv)
Standing at the forefront of knowledge dissemination, digital libraries curate vast collections of scientific literature. However, these scholarly writings are often laden with jargon and tailored for domain experts rather than the general public. As librarians, we strive to offer services to a diverse audience, including those with lower reading levels. To extend our services beyond mere access, we propose fine-tuning a language model to rewrite scholarly abstracts into more comprehensible versions, thereby making scholarly literature more accessible when requested. We began by introducing a corpus specifically designed for training models to simplify scholarly abstracts. This corpus consists of over three thousand pairs of abstracts and significance statements from diverse disciplines. We then fine-tuned four language models using this corpus. The outputs from the models were subsequently examined both quantitatively for accessibility and semantic coherence, and qualitatively for language quality, faithfulness, and completeness. Our findings show that the resulting models can improve readability by over three grade levels, while maintaining fidelity to the original content. Although commercial state-of-the-art models still hold an edge, our models are much more compact, can be deployed locally in an affordable manner, and alleviate the privacy concerns associated with using commercial models. We envision this work as a step toward more inclusive and accessible libraries, improving our services for young readers and those without a college degree.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 36 references that are in the map. First posted on arXiv 7 Aug 2024.
Builds on
17 of 36 references.
References · 17
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
- 2024OLMo: Accelerating the Science of Language Modelscited by 599
- 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
5 more
- 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
- 2022Making Science Simple: Corpora for the Lay Summarisation of Scientific Literaturecited by 54
- 2021Paragraph-level Simplification of Medical Textscited by 52
- 2018Integrating Transformer and Paraphrase Rules for Sentence Simplificationcited by 29
- 2017Evaluating prose style transfer with the Biblecited by 23
Built on by
Citing papers per year: 2024: 1.
1 paper.