mlsanityData to 14 Oct 2026

Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context

arXiv 2607.05880 ↗First posted 7 Jul 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment alone. The most direct opportunity is reducing the time and effort radiologists spend producing reports, a task that requires interpreting images, integrating clinical history and prior studies, and drafting structured findings. We present Harrison.Rad 1.5 (HR1.5), a radiology-specific multimodal large language model that accepts interleaved text and visual inputs and generates structured and unstructured text across plain-film radiology, spanning computed radiography, chest, musculoskeletal, abdominal, spine, and pelvic x-rays, and mammography. HR1.5 is trained through a three-stage pipeline: domain adaptation of a base language model on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives on ~6 million image-report instances, and visual-question-answering fine-tuning on multi-turn conversations. We evaluate it with a Findings-Diagnosis scoring framework that extends RadGraph-XL entity extraction with ontology-based synonym matching and polarity-contradiction detection, benchmarked on RadBench, a simulated FRCR 2B Short Case examination scored against Angoff-method thresholds, ReXGradient, and internal multi-modality datasets. HR1.5 is the only system evaluated to meet the simulated FRCR passing standard and achieves the highest accuracy on closed-format clinical questions, across anatomical regions, on internal multi-body-part and mammography reporting, and on the primary clinically-aligned score for public chest reporting. We further examine explainability and model behaviour, including question-sensitive Grad-CAM heatmaps, attention analysis, and confidence estimation, to support responsible future evaluation toward clinical use, and a framework for clinically grounded assessment of report quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 73 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

40 of 73 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2017On Calibration of Modern Neural Networkscited by 4,960
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  11. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  12. 2024A Survey on LLM-as-a-Judgecited by 1,423
28 more
  1. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2019Attention is not Explanationcited by 929
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  9. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  10. 2019Attention is not not Explanationcited by 636
  11. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  12. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  13. 2025MedGemma Technical Reportcited by 436
  14. 2024Capabilities of Gemini Models in Medicinecited by 237
  15. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
  16. 2016Generating Visual Explanationscited by 158
  17. 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
  18. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 135
  19. 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
  20. 2024A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretationcited by 81
  21. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 74
  22. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 74
  23. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
  24. 2025ReXGradient-160K: A Large-Scale Publicly Available Dataset of Chest Radiographs with Free-text Reportscited by 27
  25. 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 27
  26. 2024Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generationcited by 10
  27. 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretationcited by 5
  28. 2026CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
  2. 2026RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
  3. 2025RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment
  4. 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiology
  5. 2025RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction
  6. 2026RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
  7. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation
  8. 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement