# The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces

arXiv: https://arxiv.org/abs/2303.14334 · HTML: https://mlsanity.com/paper/2303.14334/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 119 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers](https://mlsanity.com/paper/2012.15828/) (2020)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols](https://mlsanity.com/paper/2009.14237/) (2020)
- [Pre-trained Language Model for Biomedical Question Answering](https://mlsanity.com/paper/1909.08229/) (2019)
- [VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups](https://mlsanity.com/paper/2106.00676/) (2021)
- [Scim: Intelligent Skimming Support for Scientific Papers](https://mlsanity.com/paper/2205.04561/) (2022)
- [Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing](https://mlsanity.com/paper/2203.00130/) (2022)
- [Threddy: An Interactive System for Personalized Thread-based Exploration and Organization of Scientific Literature](https://mlsanity.com/paper/2208.03455/) (2022)
- [CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Context](https://mlsanity.com/paper/2302.07302/) (2023)
- [Relatedly: Scaffolding Literature Reviews with Existing Related Work Sections](https://mlsanity.com/paper/2302.06754/) (2023)
- [Bursting Scientific Filter Bubbles: Boosting Innovation via Novel Author Discovery](https://mlsanity.com/paper/2108.05669/) (2021)
- [Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions](https://mlsanity.com/paper/2010.05129/) (2020)
- [A Dataset of Alt Texts from HCI Publications: Analyses and Uses Towards Producing More Descriptive Alt Texts of Data Visualizations in Scientific Papers](https://mlsanity.com/paper/2209.13718/) (2022)
- [Kori: Interactive Synthesis of Text and Charts in Data Documents](https://mlsanity.com/paper/2108.04203/) (2021)
- [ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts](https://mlsanity.com/paper/2205.06982/) (2022)
- [From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks](https://mlsanity.com/paper/2204.10254/) (2022)
- [Scientific Article Recommendation: Exploiting Common Author Relations and Historical Preferences](https://mlsanity.com/paper/2008.04652/) (2020)
- [Improving the Accessibility of Scientific Documents: Current State, User Needs, and a System Solution to Enhance Scientific PDF Accessibility for Blind and Low Vision Users](https://mlsanity.com/paper/2105.00076/) (2021)

## Built on by (most cited first, 9 of 9)

- [Language agents achieve superhuman synthesis of scientific knowledge](https://mlsanity.com/paper/2409.13740/) (2024)
- [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025)
- [LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions](https://mlsanity.com/paper/2411.05025/) (2024)
- [PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers](https://mlsanity.com/paper/2403.02939/) (2024)
- [SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers](https://mlsanity.com/paper/2411.05338/) (2024)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [Citance-Contextualized Summarization of Scientific Papers](https://mlsanity.com/paper/2311.02408/) (2023)
- [From Nuisance to News Sense: Augmenting the News with Cross-Document Evidence and Context](https://mlsanity.com/paper/2310.04592/) (2023)
- [Robust Multilingual Text-to-Pictogram Mapping for Scalable Reading Rehabilitation](https://mlsanity.com/paper/2603.24536/) (2026)

## Abstract (arXiv)

Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading process grows. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. The PDF format for sharing research papers is widely used due to its portability, but it has significant downsides including: static content, poor accessibility for low-vision readers, and difficulty reading on mobile devices. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces -- even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed ten research prototype interfaces and conducted usability studies with more than 300 participants and real-world users showing improved reading experiences for scholars. We've also released a production reading interface for research papers that will incorporate the best features as they mature. We structure this paper around challenges scholars and the public face when reading research papers -- Discovery, Efficiency, Comprehension, Synthesis, and Accessibility -- and present an overview of our progress and remaining open challenges.