# SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations

arXiv: https://arxiv.org/abs/2509.20567 · HTML: https://mlsanity.com/paper/2509.20567/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 24 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Towards Accurate Differential Diagnosis with Large Language Models](https://mlsanity.com/paper/2312.00164/) (2023)
- [KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniques](https://mlsanity.com/paper/2403.05881/) (2024)
- [Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales](https://mlsanity.com/paper/2312.07399/) (2023)
- [Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective](https://mlsanity.com/paper/2411.14258/) (2024)
- [Large Language Models Perform Diagnostic Reasoning](https://mlsanity.com/paper/2307.08922/) (2023)
- [From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?](https://mlsanity.com/paper/2403.11894/) (2024)
- [Digital Diagnostics: The Potential Of Large Language Models In Recognizing Symptoms Of Common Illnesses](https://mlsanity.com/paper/2405.06712/) (2024)
- [From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice](https://mlsanity.com/paper/2410.01812/) (2024)
- [Large Language Models for Interpretable Mental Health Diagnosis](https://mlsanity.com/paper/2501.07653/) (2025)
- [Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning](https://mlsanity.com/paper/2306.04551/) (2023)
- [Interpretable Differential Diagnosis with Dual-Inference Large Language Models](https://mlsanity.com/paper/2407.07330/) (2024)
- [An Analysis on Large Language Models in Healthcare: A Case Study of BioBERT](https://mlsanity.com/paper/2310.07282/) (2023)
- [MedG-KRP: Medical Graph Knowledge Representation Probing](https://mlsanity.com/paper/2412.10982/) (2024)
- [From Hallucinations to Facts: Enhancing Language Models with Curated Knowledge Graphs](https://mlsanity.com/paper/2412.18672/) (2024)

## Abstract (arXiv)

In multilingual healthcare environments, automatic disease diagnosis from clinical text remains a challenging task due to the scarcity of annotated medical data in low-resource languages and the linguistic variability across populations. This paper proposes SwasthLLM, a unified, zero-shot, cross-lingual, and multi-task learning framework for medical diagnosis that operates effectively across English, Hindi, and Bengali without requiring language-specific fine-tuning. At its core, SwasthLLM leverages the multilingual XLM-RoBERTa encoder augmented with a language-aware attention mechanism and a disease classification head, enabling the model to extract medically relevant information regardless of the language structure. To align semantic representations across languages, a Siamese contrastive learning module is introduced, ensuring that equivalent medical texts in different languages produce similar embeddings. Further, a translation consistency module and a contrastive projection head reinforce language-invariant representation learning. SwasthLLM is trained using a multi-task learning strategy, jointly optimizing disease classification, translation alignment, and contrastive learning objectives. Additionally, we employ Model-Agnostic Meta-Learning (MAML) to equip the model with rapid adaptation capabilities for unseen languages or tasks with minimal data. Our phased training pipeline emphasizes robust representation alignment before task-specific fine-tuning. Extensive evaluation shows that SwasthLLM achieves high diagnostic performance, with a test accuracy of 97.22% and an F1-score of 97.17% in supervised settings. Crucially, in zero-shot scenarios, it attains 92.78% accuracy on Hindi and 73.33% accuracy on Bengali medical text, demonstrating strong generalization in low-resource contexts.