# Large Language Models for Scientific Information Extraction: An Empirical Study for Virology

arXiv: https://arxiv.org/abs/2401.10040 · HTML: https://mlsanity.com/paper/2401.10040/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 65 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://mlsanity.com/paper/1806.08730/) (2018)
- [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://mlsanity.com/paper/2101.11038/) (2021)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [DART: Open-Domain Structured Data Record to Text Generation](https://mlsanity.com/paper/2007.02871/) (2020)
- [Automated Mining of Leaderboards for Empirical AI Research](https://mlsanity.com/paper/2109.13089/) (2021)
- [ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graph](https://mlsanity.com/paper/2305.11068/) (2023)
- [Zero-shot Entailment of Leaderboards for Empirical AI Research](https://mlsanity.com/paper/2303.16835/) (2023)

## Built on by (most cited first, 12 of 12)

- [Automatically Extracting Numerical Results from Randomized Controlled Trials with Large Language Models](https://mlsanity.com/paper/2405.01686/) (2024)
- [34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery](https://mlsanity.com/paper/2505.03049/) (2025)
- [Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry](https://mlsanity.com/paper/2411.15221/) (2024)
- [Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models](https://mlsanity.com/paper/2501.18287/) (2025)
- [Contri(e)ve: Context + Retrieve for Scholarly Question Answering](https://mlsanity.com/paper/2409.09010/) (2024)
- [From Keywords to Structured Summaries: Streamlining Scholarly Information Access](https://mlsanity.com/paper/2402.14622/) (2024)
- [Towards Global AI Inclusivity: A Large-Scale Multilingual Terminology Dataset (GIST)](https://mlsanity.com/paper/2412.18367/) (2024)
- [QoS-QoE Translation with Large Language Model](https://mlsanity.com/paper/2604.08703/) (2026)
- [Exploring the Latest LLMs for Leaderboard Extraction](https://mlsanity.com/paper/2406.04383/) (2024)
- [The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale](https://mlsanity.com/paper/2605.15011/) (2026)
- [SciSchema.org: A Multidisciplinary Collection of Schemas for Structured Scientific Process Descriptions](https://mlsanity.com/paper/2607.27955/) (2026)
- [From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction](https://mlsanity.com/paper/2609.34829/) (2026)

## Abstract (arXiv)

In this paper, we champion the use of structured and semantic content representation of discourse-based scholarly communication, inspired by tools like Wikipedia infoboxes or structured Amazon product descriptions. These representations provide users with a concise overview, aiding scientists in navigating the dense academic landscape. Our novel automated approach leverages the robust text generation capabilities of LLMs to produce structured scholarly contribution summaries, offering both a practical solution and insights into LLMs' emergent abilities. For LLMs, the prime focus is on improving their general intelligence as conversational agents. We argue that these models can also be applied effectively in information extraction (IE), specifically in complex IE tasks within terse domains like Science. This paradigm shift replaces the traditional modular, pipelined machine learning approach with a simpler objective expressed through instructions. Our results show that finetuned FLAN-T5 with 1000x fewer parameters than the state-of-the-art GPT-davinci is competitive for the task.