# Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models

arXiv: https://arxiv.org/abs/2604.01965 · HTML: https://mlsanity.com/paper/2604.01965/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs](https://mlsanity.com/paper/2411.14199/) (2024)
- [SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis](https://mlsanity.com/paper/2403.01976/) (2024)
- [SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature](https://mlsanity.com/paper/2406.07835/) (2024)
- [Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains](https://mlsanity.com/paper/2402.05140/) (2024)
- [unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network](https://mlsanity.com/paper/2303.14957/) (2023)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [APPLS: Evaluating Evaluation Metrics for Plain Language Summarization](https://mlsanity.com/paper/2305.14341/) (2023)
- [Know Your Audience: The benefits and pitfalls of generating plain language summaries beyond the "general" audience](https://mlsanity.com/paper/2403.04979/) (2024)
- [SemOpenAlex: The Scientific Landscape in 26 Billion RDF Triples](https://mlsanity.com/paper/2308.03671/) (2023)
- [Question Answering on Scholarly Knowledge Graphs](https://mlsanity.com/paper/2006.01527/) (2020)
- [Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation](https://mlsanity.com/paper/2012.02420/) (2020)
- [ROUGE-K: Do Your Summaries Have Keywords?](https://mlsanity.com/paper/2403.05186/) (2024)

## Built on by (most cited first, 1 of 1)

- [The zbMATH Open Knowledge Graph: Tracing Centuries of Mathematical Research](https://mlsanity.com/paper/2609.00969/) (2026)

## Abstract (arXiv)

Scientific knowledge discovery increasingly relies on large language models, yet many existing scholarly assistants depend on proprietary systems with tens or hundreds of billions of parameters. Such reliance limits reproducibility and accessibility for the research community. In this work, we ask a simple question: do we need bigger models for scientific applications? Specifically, we investigate to what extent carefully designed retrieval pipelines can compensate for reduced model scale in scientific applications. We design a lightweight retrieval-augmented framework that performs task-aware routing to select specialized retrieval strategies based on the input query. The system further integrates evidence from full-text scientific papers and structured scholarly metadata, and employs compact instruction-tuned language models to generate responses with citations. We evaluate the framework across several scholarly tasks, focusing on scholarly question answering (QA), including single- and multi-document scenarios, as well as biomedical QA under domain shift and scientific text compression. Our findings demonstrate that retrieval and model scale are complementary rather than interchangeable. While retrieval design can partially compensate for smaller models, model capacity remains important for complex reasoning tasks. This work highlights retrieval and task-aware design as key factors for building practical and reproducible scholarly assistants.