mlsanityData to 7 Oct 2026

Neurosymbolic Alignment for Physiologically-Safe Clinical Language Models

arXiv 2608.24534 ↗First posted 25 Aug 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Clinical LLMs can generate recommendations that are factually plausible yet physiologically unsafe. We investigate whether safety alignment can be improved by grounding preference optimization in structured physiological knowledge rather than text-only supervision. Methods: We propose Neurosymbolic Alignment, a training-time framework that couples a 7B clinical LLM with an HGNN-based Physiological World Model over an 847K-node biomedical knowledge graph. Candidate responses are scored using homeostatic constraints, multi-hop path plausibility, and drug-interaction penalties, and the resulting rankings drive iterative on-policy ORPO updates. Evaluation is performed on the Clinical Safety Benchmark (CSB), a 2,500-scenario benchmark for physiological constraint violations in generative clinical reasoning. Results: Relative to ORPO, the proposed method improves CSS from 69.5% to 90.8% (+21.3 pp), reduces physician-evaluated HR from 14.1% to 5.1% on the blinded subset, and improves DID from 72.8% to 91.6%. These gains are corroborated by an HGNN-independent Rule-Engine Safety Score (RSS: 86.4%, +21.2 pp over ORPO; r=0.97 concordance with CSS). The method also exceeds GPT-4 (5-shot) on all safety metrics despite a 10x parameter disadvantage, and outperforms an inference-time self-correction pipeline (SFT+SelfCorrect) by 11.4 pp CSS. Under synthetic EHR-style noise, 84.2% CSS is retained. Ablation analysis shows that HGNN scoring (-16.2 pp) and iterative training (-11.5 pp) are the dominant contributors. PhysioScore calibration against 200 clinician labels yielded ECE = 0.038 and kappa = 0.91. Conclusion: Training-time physiological grounding produces measurable and independently verifiable safety improvements in open-weight clinical LLMs under controlled evaluation. External validation on real clinical data is required to determine whether these gains transfer to deployment settings

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 27 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

15 of 27 references.

References · 15

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  5. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  6. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  7. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  8. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  9. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 346
  10. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  11. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  12. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
3 more
  1. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  2. 2023GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphscited by 57
  3. 2025Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AIcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Black-Box Behavioral Distillation Breaks Safety Alignment in Medical LLMs
  2. 2025The Forgotten Shield: Safety Grafting in Parameter-Space for Medical MLLMs
  3. 2026ProMedical: Hierarchical Fine-Grained Criteria Modeling for Medical LLM Alignment via Explicit Injection
  4. 2026When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries
  5. 2025Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies
  6. 2026Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
  7. 2024MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
  8. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Models