# Learning to Fuse LLMs with Ontology Rankers for Rare-Disease Diagnosis

arXiv: https://arxiv.org/abs/2609.02473 · HTML: https://mlsanity.com/paper/2609.02473/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Builds on (most cited first, 13 of 13)

- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [When Does Confidence-Based Cascade Deferral Suffice?](https://mlsanity.com/paper/2307.02764/) (2023)
- [An Agentic System for Rare Disease Diagnosis with Traceable Reasoning](https://mlsanity.com/paper/2506.20430/) (2025)
- [An Analysis of Fusion Functions for Hybrid Retrieval](https://mlsanity.com/paper/2210.11934/) (2022)
- [RareBench: Can LLMs Serve as Rare Diseases Specialists?](https://mlsanity.com/paper/2402.06341/) (2024)
- [Smoothie: Label Free Language Model Routing](https://mlsanity.com/paper/2412.04692/) (2024)
- [Search-Time Contamination in Deep Research Agents: Measuring Performance Inflation in Public Benchmark Evaluation](https://mlsanity.com/paper/2606.05241/) (2026)
- [LiveClin: A Live Clinical Benchmark without Leakage](https://mlsanity.com/paper/2602.16747/) (2026)
- [ProbFuse: A Probabilistic Approach to Data Fusion](https://mlsanity.com/paper/1409.8518/) (2014)
- [LA-MARRVEL: A Knowledge-Grounded, Language-Aware LLM Framework for Clinically Robust Rare Disease Gene Prioritization](https://mlsanity.com/paper/2511.02263/) (2025)
- [L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification](https://mlsanity.com/paper/2604.13285/) (2026)

## Abstract (arXiv)

Ontology rankers remain useful for rare-disease diagnosis because each candidate can be traced to matched patient phenotypes. Large language models (LLMs) can generate differential diagnoses from the same patient description, but their predictions lack an equally clear evidence trail. Rather than asking which system should replace the other, we ask whether an LLM can improve the ranker without giving up its evidence. Our behavior-based fusion model examines the two ranked lists, their agreement, and the ontology support behind each candidate, and learns how much to rely on each system for the individual case. Before comparison, we remove a documented test-set leakage pathway caused by benchmark cases and ontology annotations being derived from the same publications. Across eight open LLMs, fusion improves Phenomizer Recall@1 by 7.86 percentage points on Phenopacket Store and 20.18 points on RAMEDIS. When paired with DeepSeek-V4-Flash through an API, a fusion model trained only on the other LLMs improves Recall@1 from 0.1657 to 0.2176, a 5.19-point gain, without retraining. For 90.8% of correct fused diagnoses, the disease retains candidate-level ontology evidence that can be inspected. These results show that LLMs can strengthen an established diagnostic tool without discarding the structured evidence that makes it useful.