mlsanityData to 14 Oct 2026

TreeProbe : A Tibetan Medicine Benchmark for Cultural Bias in LLMs

arXiv 2608.00640 ↗First posted 1 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are increasingly viewed as a potential means of mitigating global health inequities, yet their outputs often reflect dominant high-resource medical traditions and provide limited coverage of traditional medical knowledge systems. Tibetan medicine, one of the world's four major traditional medical systems, has an independent and highly structured theoretical framework. When models lack grounded understanding of Tibetan medicine, they may fall back on dominant epistemic systems and distort the native knowledge structure during reasoning. However, quantitative tools for evaluating cultural bias in Tibetan medicine remain largely absent. To address this gap, we introduce TreeProbe, the first cultural-bias benchmark organized around the native Tree of Medicine framework in Tibetan medicine. It contains 4,719 expert-adjudicated items covering 467 diseases and 10 subtasks along the three roots. Experiments on representative LLMs show that current models remain limited in native Tibetan medical contexts and exhibit systematic external ontology drift. Further analysis reveals that models diverge in whether they drift toward biomedical or TCM reasoning, shaped by pretraining data composition and surface resemblance between TCM and Tibetan medicine. TreeProbe provides a diagnostic benchmark for developing medical AI systems that are both linguistically inclusive and epistemically fair. Code and data are available in an anonymous repository at https://anonymous.4open.science/r/TreeProbe/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 35 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

9 of 35 references.

References · 9

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  4. 2024DeepSeek-V3 Technical Reportcited by 811
  5. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  6. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  7. 2025LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validationcited by 34
  8. 2024TCMBench: A Comprehensive Benchmark for Evaluating Large Language Models in Traditional Chinese Medicinecited by 17
  9. 2025TCM-Eval: An Expert-Level Dynamic and Extensible Benchmark for Traditional Chinese Medicinecited by 2

Built on by

Not cited yet.