mlsanityData to 7 Oct 2026

Can We Edit LLMs for Long-Tail Biomedical Knowledge?

arXiv 2504.10421 ↗First posted 14 Apr 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Knowledge editing has emerged as an effective approach for updating large language models (LLMs) by modifying their internal knowledge. However, their application to the biomedical domain faces unique challenges due to the long-tailed distribution of biomedical knowledge, where rare and infrequent information is prevalent. In this paper, we conduct the first comprehensive study to investigate the effectiveness of knowledge editing methods for editing long-tail biomedical knowledge. Our results indicate that, while existing editing methods can enhance LLMs' performance on long-tail biomedical knowledge, their performance on long-tail knowledge remains inferior to that on high-frequency popular knowledge, even after editing. Our further analysis reveals that long-tail biomedical knowledge contains a significant amount of one-to-many knowledge, where one subject and relation link to multiple objects. This high prevalence of one-to-many knowledge limits the effectiveness of knowledge editing in improving LLMs' understanding of long-tail biomedical knowledge, highlighting the need for tailored strategies to bridge this performance gap.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 58 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

28 of 58 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  6. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  7. 2022Mass-Editing Memory in a Transformercited by 976
  8. 2021Editing Factual Knowledge in Language Modelscited by 548
  9. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  10. 2023The Falcon Series of Open Language Modelscited by 526
  11. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  12. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
16 more
  1. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  2. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  3. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  4. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  5. 2023PMET: Precise Model Editing in a Transformercited by 170
  6. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  7. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  8. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  9. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 78
  10. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  11. 2022Knowledge Graph Embedding: A Survey from the Perspective of Representation Spacescited by 41
  12. 2022ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Datacited by 34
  13. 2020Biomedical named entity recognition using BERT in the machine reading comprehension frameworkcited by 22
  14. 2024Graph-Based Retriever Captures the Long Tail of Biomedical Knowledgecited by 20
  15. 2021Probing Pre-Trained Language Models for Disease Knowledgecited by 10
  16. 2024From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answeringcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editingcited by 4
  2. 2025From Memorization to Generalization: Fine-Tuning Large Language Models for Biomedical Term-to-Identifier Normalizationcited by 1
  3. 2026Latent Knowledge as a Predictor of Fact Acquisition in Fine-Tuned Large Language Modelscited by 0