# LOTUS: Learning to Optimize Task-based US representations

arXiv: https://arxiv.org/abs/2307.16021 · HTML: https://mlsanity.com/paper/2307.16021/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 29 Jul 2023.

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss](https://mlsanity.com/paper/1804.10916/) (2018)
- [Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation](https://mlsanity.com/paper/1806.07201/) (2018)
- [Tracked 3D Ultrasound and Deep Neural Network-based Thyroid Segmentation reduce Interobserver Variability in Thyroid Volumetry](https://mlsanity.com/paper/2108.10118/) (2021)
- [Rethinking Ultrasound Augmentation: A Physics-Inspired Approach](https://mlsanity.com/paper/2105.02188/) (2021)
- [Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images](https://mlsanity.com/paper/2103.05745/) (2021)
- [CACTUSS: Common Anatomical CT-US Space for US examinations](https://mlsanity.com/paper/2207.08619/) (2022)
- [Investigating Pulse-Echo Sound Speed Estimation in Breast Ultrasound with Deep Learning](https://mlsanity.com/paper/2302.03064/) (2023)

## Built on by (most cited first, 4 of 4)

- [Diffusion as Sound Propagation: Physics-inspired Model for Ultrasound Image Generation](https://mlsanity.com/paper/2407.05428/) (2024)
- [UltraRay: Introducing Full-Path Ray Tracing in Physics-Based Ultrasound Simulation](https://mlsanity.com/paper/2501.05828/) (2025)
- [Deep Spectral Methods for Unsupervised Ultrasound Image Interpretation](https://mlsanity.com/paper/2408.02043/) (2024)
- [DefSynUS: Real-time Patient-specific Intrahepatic Vessel Identification via Deformation-Aware CT-US Domain Adaptation](https://mlsanity.com/paper/2605.29570/) (2026)

## Abstract (arXiv)

Anatomical segmentation of organs in ultrasound images is essential to many clinical applications, particularly for diagnosis and monitoring. Existing deep neural networks require a large amount of labeled data for training in order to achieve clinically acceptable performance. Yet, in ultrasound, due to characteristic properties such as speckle and clutter, it is challenging to obtain accurate segmentation boundaries, and precise pixel-wise labeling of images is highly dependent on the expertise of physicians. In contrast, CT scans have higher resolution and improved contrast, easing organ identification. In this paper, we propose a novel approach for learning to optimize task-based ultra-sound image representations. Given annotated CT segmentation maps as a simulation medium, we model acoustic propagation through tissue via ray-casting to generate ultrasound training data. Our ultrasound simulator is fully differentiable and learns to optimize the parameters for generating physics-based ultrasound images guided by the downstream segmentation task. In addition, we train an image adaptation network between real and simulated images to achieve simultaneous image synthesis and automatic segmentation on US images in an end-to-end training setting. The proposed method is evaluated on aorta and vessel segmentation tasks and shows promising quantitative results. Furthermore, we also conduct qualitative results of optimized image representations on other organs.