# Structure-aware World Model for Probe Guidance via Large-scale Self-supervised Pre-train

arXiv: https://arxiv.org/abs/2406.19756 · HTML: https://mlsanity.com/paper/2406.19756/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Medical and surgical world models, one of 11 directions in World models.
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Placed in

- World models / [Medical and surgical world models](https://mlsanity.com/wm/medical-and-surgical-world-models/)

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Adaptive Focus for Efficient Video Recognition](https://mlsanity.com/paper/2105.03245/) (2021)
- [Cross-Modal Adapter for Vision-Language Retrieval](https://mlsanity.com/paper/2211.09623/) (2022)
- [CondenseNet V2: Sparse Feature Reactivation for Deep Networks](https://mlsanity.com/paper/2104.04382/) (2021)
- [Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding](https://mlsanity.com/paper/2203.08481/) (2022)
- [Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound](https://mlsanity.com/paper/2007.04480/) (2020)
- [Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model](https://mlsanity.com/paper/2406.13165/) (2024)
- [Multi-Modal Interaction Control of Ultrasound Scanning Robots with Safe Human Guidance and Contact Recovery](https://mlsanity.com/paper/2302.05685/) (2023)

## Built on by (most cited first, 7 of 7)

- [Surgical Vision World Model](https://mlsanity.com/paper/2503.02904/) (2025)
- [Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies](https://mlsanity.com/paper/2606.16721/) (2026)
- [UltraSeP: Sequence-aware Pre-training for Echocardiography Probe Movement Guidance](https://mlsanity.com/paper/2408.15026/) (2024)
- [VA-Adapter: Adapting Ultrasound Foundation Model to Echocardiography Probe Guidance](https://mlsanity.com/paper/2510.06809/) (2025)
- [Echo-POSED: Geometric Self-Distillation for Echocardiography Guidance](https://mlsanity.com/paper/2606.02634/) (2026)
- [UltraStar: Semantic-Aware Star Graph Modeling for Echocardiography Navigation](https://mlsanity.com/paper/2603.01461/) (2026)
- [SonoWorld: Towards Interactive Pose-Controllable Ultrasound World Models](https://mlsanity.com/paper/2610.19327/) (2026)

## Abstract (arXiv)

The complex structure of the heart leads to significant challenges in echocardiography, especially in acquisition cardiac ultrasound images. Successful echocardiography requires a thorough understanding of the structures on the two-dimensional plane and the spatial relationships between planes in three-dimensional space. In this paper, we innovatively propose a large-scale self-supervised pre-training method to acquire a cardiac structure-aware world model. The core innovation lies in constructing a self-supervised task that requires structural inference by predicting masked structures on a 2D plane and imagining another plane based on pose transformation in 3D space. To support large-scale pre-training, we collected over 1.36 million echocardiograms from ten standard views, along with their 3D spatial poses. In the downstream probe guidance task, we demonstrate that our pre-trained model consistently reduces guidance errors across the ten most common standard views on the test set with 0.29 million samples from 74 routine clinical scans, indicating that structure-aware pre-training benefits the scanning.