mlsanityData to 7 Oct 2026

RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements

arXiv 2411.15700 ↗First posted 24 Nov 2024cs.CL cs.AI cs.CEcited by 11

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Objective: We aimed to develop an advanced multi-task large language model (LLM) framework to extract multiple types of information about dietary supplements (DS) from clinical records. Methods: We used four core DS information extraction tasks - namely, named entity recognition (NER: 2,949 clinical sentences), relation extraction (RE: 4,892 sentences), triple extraction (TE: 2,949 sentences), and usage classification (UC: 2,460 sentences) as our multitasks. We introduced a novel Retrieval-Augmented Multi-task Information Extraction (RAMIE) Framework, including: 1) employed instruction fine-tuning techniques with task-specific prompts, 2) trained LLMs for multiple tasks with improved storage efficiency and lower training costs, and 3) incorporated retrieval augmentation generation (RAG) techniques by retrieving similar examples from the training set. We compared RAMIE's performance to LLMs with instruction fine-tuning alone and conducted an ablation study to assess the contributions of multi-task learning and RAG to improved multitasking performance. Results: With the aid of the RAMIE framework, Llama2-13B achieved an F1 score of 87.39 (3.51% improvement) on the NER task and demonstrated outstanding performance on the RE task with an F1 score of 93.74 (1.15% improvement). For the TE task, Llama2-7B scored 79.45 (14.26% improvement), and MedAlpaca-7B achieved the highest F1 score of 93.45 (0.94% improvement) on the UC task. The ablation study revealed that while MTL increased efficiency with a slight trade-off in performance, RAG significantly boosted overall accuracy. Conclusion: This study presents a novel RAMIE framework that demonstrates substantial improvements in multi-task information extraction for DS-related data from clinical records. Our framework can potentially be applied to other domains.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 59 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

23 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Benchmarking Retrieval-Augmented Large Language Models in Biomedical NLP: Application, Robustness, and Self-Awareness13 May

Other references · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Mistral 7Bcited by 3,065
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  7. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  8. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  9. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  10. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
  11. 2017Joint Extraction of Entities and Relations Based on a Novel Tagging Schemecited by 102
  12. 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
10 more
  1. 2024STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Basescited by 53
  2. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  3. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  4. 2024Pushing The Limit of LLM Capacity for Text Classificationcited by 28
  5. 2020MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learningcited by 17
  6. 2024Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extractioncited by 14
  7. 2024LLMs in Biomedicine: A study on clinical Named Entity Recognitioncited by 12
  8. 2020Deep Learning Approaches for Extracting Adverse Events and Indications of Dietary Supplements from Clinical Textcited by 3
  9. 2021A Conversational Agent System for Dietary Supplements Usecited by 2
  10. 2021Discovering novel drug-supplement interactions using a dietary supplements knowledge graph generated from the biomedical literaturecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2025MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learningcited by 13
  2. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  3. 2024Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Modelscited by 6
  4. 2025An evaluation of DeepSeek Models in Biomedical Natural Language Processingcited by 6
  5. 2025Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendationcited by 3
  6. 2026Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Studycited by 2
  7. 2025EPEE: Towards Efficient and Effective Foundation Models in Biomedicinecited by 1
  8. 2025Inference Gap in Domain Expertise and Machine Intelligence in Named Entity Recognition: Creation of and Insights from a Substance Use-related Datasetcited by 0
  9. 2026An Underexplored Frontier: Large Language Models for Rare Disease Patient Education and Communication -- A scoping reviewcited by 0
  10. 2026Retrieval-Augmented Large Language Models for Schema-Constrained Clinical Information Extractioncited by 0
  11. 2026ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluation of LLMs in retrieving food and nutritional context for RAG systems
  2. 2026Harnessing Large Language Models for Precision Querying and Retrieval-Augmented Knowledge Extraction in Clinical Data Science
  3. 2024A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences
  4. 2026TSWAP: A Multilingual Retrieval-Augmented Thai Wellness Advisor
  5. 2024BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine
  6. 2024LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction
  7. 2026Relation Extraction Capabilities of LLMs on Clinical Text: A Bilingual Evaluation for English and Turkish
  8. 2024LLM-IE: A Python Package for Generative Information Extraction with Large Language Models