mlsanityData to 14 Oct 2026

Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning

arXiv 2407.14904 ↗First posted 20 Jul 2024eess.IV cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16 million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 48 of its 100 references that are in the map. First posted on arXiv 20 Jul 2024.

Builds on

48 of 100 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2020Denoising Diffusion Implicit Modelscited by 9,781
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  11. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  12. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
36 more
  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  11. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  13. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  14. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  15. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  16. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  17. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  18. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  19. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
  20. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  21. 2021XCiT: Cross-Covariance Image Transformerscited by 338
  22. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  23. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 194
  24. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 150
  25. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  26. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
  27. 2022Perceptual Grouping in Contrastive Vision-Language Modelscited by 82
  28. 2024Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathologycited by 72
  29. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 66
  30. 2023A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticscited by 43
  31. 2023CDUL: CLIP-Driven Unsupervised Learning for Multi-Label Image Classificationcited by 39
  32. 2023Connecting Multi-modal Contrastive Representationscited by 36
  33. 2022Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Imagescited by 32
  34. 2024Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interactioncited by 27
  35. 2020Divide-and-Rule: Self-Supervised Learning for Survival Analysis in Colorectal Cancercited by 17
  36. 2023Cross-scale Multi-instance Learning for Pathological Image Diagnosiscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025FEAT: A Multi-Agent Forensic AI System with Domain-Adapted Large Language Model for Automated Cause-of-Death Analysiscited by 1
  2. 2026Through the Schr"odinger Bridge: Benchmarking Antemortem Image Restoration from Postmortem Autolysis to Enhance Forensic Diagnosticscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?
  2. 2026Enhancing Pathological VLMs with Cross-scale Reasoning
  3. 2025PathoHR: Hierarchical Reasoning for Vision-Language Models in Pathology
  4. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
  5. 2025PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks
  6. 2024Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis
  7. 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model
  8. 2025Multi-Modal Foundation Models for Computational Pathology: A Survey