# UltraPIPS: Improving model perception in B-mode ultrasound with foundation models

arXiv: https://arxiv.org/abs/2608.26033 · HTML: https://mlsanity.com/paper/2608.26033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 26 Aug 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis](https://mlsanity.com/paper/2401.00153/) (2023)
- [MicroSegNet: A Deep Learning Approach for Prostate Segmentation on Micro-Ultrasound Images](https://mlsanity.com/paper/2305.19956/) (2023)
- [UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets](https://mlsanity.com/paper/2411.16222/) (2024)
- [EchoPrime: A Multi-Video View-Informed Vision-Language Model for Comprehensive Echocardiography Interpretation](https://mlsanity.com/paper/2410.09704/) (2024)
- [Speckle2Self: Self-Supervised Ultrasound Speckle Reduction Without Clean Data](https://mlsanity.com/paper/2507.06828/) (2025)
- [Ultrasound-CLIP: Semantic-Aware Contrastive Pre-training for Ultrasound Image-Text Understanding](https://mlsanity.com/paper/2604.01749/) (2026)
- [Generative augmentations for improved cardiac ultrasound segmentation using diffusion models](https://mlsanity.com/paper/2502.20100/) (2025)
- [X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classification](https://mlsanity.com/paper/2406.16109/) (2024)
- [Pre-trained Models Succeed in Medical Imaging with Representation Similarity Degradation](https://mlsanity.com/paper/2503.07958/) (2025)
- [High Volume Rate 3D Ultrasound Reconstruction with Diffusion Models](https://mlsanity.com/paper/2505.22090/) (2025)
- [A texture-based framework for foundational ultrasound models](https://mlsanity.com/paper/2602.01444/) (2026)

## Abstract (arXiv)

In medical imaging, it is common to use learned perceptual image patch similarity (LPIPS) to compare images semantically in feature space. Although backbones pretrained on natural images are widely used for LPIPS computation, B-mode ultrasound images possess distinct speckle patterns and acoustic-specific image statistics that are fundamentally different from natural images and even from other images in radiology. Consequently, we propose that domain-specific models are needed to measure perceptual similarity in ultrasound data, a finding which is not necessarily the case for other imaging modalities. We compare LPIPS metrics across downstream tasks like classification, segmentation and reconstruction using natural image, medical generalist and ultrasound backbone models and show that selection of LPIPS backbone is a non-trivial design choice. In particular, the ultrasound backbone models were more correlated with downstream performance of supervised models than classical and natural image models, and optimization of the LPIPS loss with an ultrasound backbone achieved a strong balance between reconstruction quality and realism. Our code is available at https://github.com/talg2324/UltraPIPS and introduces the UltraPIPS library, a set of LPIPS metrics based on the open-source foundation models analyzed in this paper.