mlsanityData to 7 Oct 2026

SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature

arXiv 2406.07835 ↗First posted 10 Jun 2024cs.CL cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following instances for training and evaluation, covering 54 tasks. These tasks span five core scientific literature understanding capabilities: information extraction, summarization, question answering, claim verification, and classification. SciRIFF is unique in being entirely expert-written, high-quality instruction-following dataset for extracting and synthesizing information from research literature across diverse scientific fields. It features complex instructions with long input contexts, detailed task descriptions, and structured outputs. To demonstrate its utility, we finetune a series of large language models (LLMs) using a mix of general-domain and SciRIFF instructions. On nine out-of-distribution held-out tasks (referred to as SciRIFF-Eval), LLMs finetuned on SciRIFF achieve 70.6% average improvement over baselines trained only on general-domain instructions. SciRIFF facilitates the development and evaluation of LLMs to help researchers navigate the rapidly growing body of scientific literature.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 86 of its 122 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

86 of 122 references.

References · 86

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
38 more
  1. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  3. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  4. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  5. 2022Galactica: A Large Language Model for Sciencecited by 736
  6. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  7. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  8. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  9. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  10. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  11. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  12. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  13. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  14. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  15. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
  16. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  17. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  18. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  19. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  20. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  21. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  22. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  23. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 167
  24. 2023Self-Alignment with Instruction Backtranslationcited by 149
  25. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  26. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  27. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
  28. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  29. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
  30. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 131
  31. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
  32. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  33. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 124
  34. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 112
  35. 2021COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemiccited by 100
  36. 2023Revisiting Relation Extraction in the era of Large Language Modelscited by 100
  37. 2024ChemLLM: A Chemical Large Language Modelcited by 100
  38. 2020Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articlescited by 96

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 17; 2026: 15.

37 papers.

Sort:

By area:

  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  2. 2025Survey on Evaluation of LLM-based Agentscited by 190
  3. 2025ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoningcited by 41
  4. 2025Large-Scale Data Selection for Instruction Tuningcited by 34
  5. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Researchcited by 30
  6. 2024SciLitLLM: How to Adapt LLMs for Scientific Literature Understandingcited by 29
  7. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  8. 2025Sequential Diffusion Language Modelscited by 19
  9. 2025The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authorscited by 16
  10. 2025Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMscited by 14
  11. 2025Efficient Data Selection at Scale via Influence Distillationcited by 14
  12. 2025LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviewscited by 13
25 more
  1. 2025AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Researchcited by 12
  2. 2026Prescriptive Scaling Laws for Data Constrained Trainingcited by 9
  3. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  4. 2024Merge to Learn: Efficiently Adding Skills to Language Models with Model Mergingcited by 7
  5. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancementcited by 5
  6. 2026MoCo: A One-Stop Shop for Model Collaboration Researchcited by 5
  7. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoningcited by 4
  8. 2026WildSci: Advancing Scientific Reasoning from In-the-Wild Literaturecited by 4
  9. 2025Idea2Plan: Exploring AI-Powered Research Planningcited by 3
  10. 2025Improving Large Language Models with Concept-Aware Fine-Tuningcited by 2
  11. 2025Doc2SAR: A Synergistic Framework for High-Fidelity Extraction of Structure-Activity Relationships from Scientific Documentscited by 2
  12. 2025TaxoAlign: Scholarly Taxonomy Generation Using Language Modelscited by 2
  13. 2026SciMDR: Advancing Scientific Multimodal Document Reasoningcited by 2
  14. 2026Scaling Participation in Modular AI Systemscited by 2
  15. 2025The PLLuM Instruction Corpuscited by 1
  16. 2026The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systemscited by 1
  17. 2026ClaimFlow: Tracing the Evolution of Scientific Claims in NLPcited by 1
  18. 2026Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Modelscited by 1
  19. 2026From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMscited by 1
  20. 2026From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuningcited by 0
  21. 2026When Verification Fails: How Compositionally Infeasible Claims Escape Rejectioncited by 0
  22. 2026SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Modelscited by 0
  23. 2026Redesign Mixture-of-Experts Routers with Manifold Power Iterationcited by 0
  24. 2026Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMscited by 0
  25. 2026CausalMix: Data Mixture as Causal Inference for Language Model Trainingcited by 0