Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation
Not on the map. Why?
Abstract (arXiv)
Medical question answering spans specialties and clinical operations that may benefit from different adaptation directions. We propose ARBOR, a parameter-efficient method that selects rank-one components from a shared low-rank basis for each question. An additive gate combines question representations, specialty tags, operation tags, and their interaction; a learned coefficient scales the adapter residual. An illustrative separation under orthogonal, equiprobable subtasks shows how conditional selection can avoid an approximation floor faced by a fixed update with the same active rank. This result motivates the design without asserting a corresponding bound for medical corpora. On Qwen3-8B across CMB, CMExam, MedQA, and MedMCQA, five-seed experiments yield 69.69% mean accuracy across benchmarks, exceeding LoRA r16 and MoELoRA by 1.26 and 1.30 percentage points, respectively. The reported advantage over LoRA r16 increases from 0.08 to 1.94 points as training expands from one to seven specialties. Tag perturbations and atom masking support the usefulness of clinical routing, while atom clusters align with the supplied specialty labels (adjusted Rand index 0.62). Calibration, transfer, and measured costs further characterize the method. These findings support structured conditional adaptation for medical QA, while leaving clinical safety and broader deployment untested.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 28 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
14 of 28 references.
References · 14
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
- 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 115
- 2024MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Modelscited by 56
- 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 36
- 2025LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routingcited by 18
- 2024MeteoRA: Multiple-tasks Embedded LoRA for Large Language Modelscited by 16
- 2025Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learningcited by 13
- 2026Dual-Pathway Circuits of Object Hallucination in Vision-Language Modelscited by 7
- 2025Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuningcited by 3
- 2026Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspectivecited by 1
Built on by
Not cited yet.