Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context
Not on the map. Why?
Abstract (arXiv)
Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment alone. The most direct opportunity is reducing the time and effort radiologists spend producing reports, a task that requires interpreting images, integrating clinical history and prior studies, and drafting structured findings. We present Harrison.Rad 1.5 (HR1.5), a radiology-specific multimodal large language model that accepts interleaved text and visual inputs and generates structured and unstructured text across plain-film radiology, spanning computed radiography, chest, musculoskeletal, abdominal, spine, and pelvic x-rays, and mammography. HR1.5 is trained through a three-stage pipeline: domain adaptation of a base language model on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives on ~6 million image-report instances, and visual-question-answering fine-tuning on multi-turn conversations. We evaluate it with a Findings-Diagnosis scoring framework that extends RadGraph-XL entity extraction with ontology-based synonym matching and polarity-contradiction detection, benchmarked on RadBench, a simulated FRCR 2B Short Case examination scored against Angoff-method thresholds, ReXGradient, and internal multi-modality datasets. HR1.5 is the only system evaluated to meet the simulated FRCR passing standard and achieves the highest accuracy on closed-format clinical questions, across anatomical regions, on internal multi-body-part and mammography reporting, and on the primary clinically-aligned score for public chest reporting. We further examine explainability and model behaviour, including question-sensitive Grad-CAM heatmaps, attention analysis, and confidence estimation, to support responsible future evaluation toward clinical use, and a framework for clinically grounded assessment of report quality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 73 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
40 of 73 references.
References · 40
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Visual Instruction Tuningcited by 7,869
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2024A Survey on LLM-as-a-Judgecited by 1,423
28 more
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2019Attention is not Explanationcited by 929
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2024PaliGemma: A versatile 3B VLM for transfercited by 696
- 2019Attention is not not Explanationcited by 636
- 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
- 2025MedGemma Technical Reportcited by 436
- 2024Capabilities of Gemini Models in Medicinecited by 237
- 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
- 2016Generating Visual Explanationscited by 158
- 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
- 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 135
- 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
- 2024A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretationcited by 81
- 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 74
- 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 74
- 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
- 2025ReXGradient-160K: A Large-Scale Publicly Available Dataset of Chest Radiographs with Free-text Reportscited by 27
- 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 27
- 2024Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generationcited by 10
- 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretationcited by 5
- 2026CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting
- 2026RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
- 2025RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment
- 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiology
- 2025RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction
- 2026RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
- 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation
- 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement