mlsanityData to 14 Oct 2026

Recursive LLM Degradation in Biomedical Question Answering: A Cross-Generation Study

arXiv 2609.34257 ↗First posted 28 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Repeatedly training language models on their own generated data may create a synthetic-data feedback loop in which errors and distributional biases are reintroduced into subsequent training datasets. This paper studies that process in biomedical question answering (QA) using PubMedQA and two Qwen2.5 model sizes, 0.5B and 3B parameters. The study compares a recursive synthetic-data condition, in which generation G(k+1) is trained on answers produced by G(k), against a Human-Control condition that repeatedly uses the original human training data. The study evaluates across four generations from G0-G3 with two random seeds (42 and 123) and a fixed evaluation set of 1,000 expert-labeled samples. The evaluation includes disease and chemical entity F1, context-supported rate, lexical and semantic similarity, answer length, repetition rate, and other evaluation metrics. The Recursive condition for both model sizes and both seeds showed larger declines than the Human-Control condition in disease entity F1, chemical entity F1, context-supported rate, ROUGE-L, and cosine similarity. Under the fixed no-repeat 3-gram decoding constraint, the main observed behavioral change was increased answer length, while the measured 3-gram repetition rate did not increase. The magnitude of the difference-in-change was larger for the 3B model than for the 0.5B model. This difference was particularly apparent in disease F1, context-supported rate, cosine similarity, and answer length. These results show domain-specific changes associated with using recursive synthetic-data training in biomedical QA, but do not establish clinical hallucination rates or universal model collapse.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

17 of 20 references.

References · 17

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  5. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  6. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  7. 2023Self-Consuming Generative Models Go MADcited by 250
  8. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  9. 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 88
  10. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 70
  11. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 52
  12. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 37
5 more
  1. 2024Towards Theoretical Understandings of Self-Consuming Generative Modelscited by 29
  2. 2024How to Synthesize Text Data without Model Collapse?cited by 25
  3. 2024Rate of Model Collapse in Recursive Trainingcited by 25
  4. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 16
  5. 2025Self-Consuming Generative Models with Adversarially Curated Datacited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Robustness Against Misinformation in Biomedical Question Answering
  2. 2024A Survey for Large Language Models in Biomedicine
  3. 2024Give me Some Hard Questions: Synthetic Data Generation for Clinical QA
  4. 2024Evaluating Fine-Tuning Efficiency of Human-Inspired Learning Strategies in Medical Question Answering
  5. 2025MedBioLM: Optimizing Medical and Biological QA with Fine-Tuned Large Language Models and Retrieval-Augmented Generation
  6. 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
  7. 2026Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
  8. 2024How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions