# Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

arXiv: https://arxiv.org/abs/2108.00274 · HTML: https://mlsanity.com/paper/2108.00274/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 31 Jul 2021.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound](https://mlsanity.com/paper/2007.15273/) (2020)
- [Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning](https://mlsanity.com/paper/2006.07694/) (2020)
- [GAN-QP: A Novel GAN Framework without Gradient Vanishing and Lipschitz Constraint](https://mlsanity.com/paper/1811.07296/) (2018)

## Built on by (most cited first, 14 of 14)

- [A Review on Deep-Learning Algorithms for Fetal Ultrasound-Image Analysis](https://mlsanity.com/paper/2201.12260/) (2022)
- [Deep Motion Network for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2207.00177/) (2022)
- [Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames](https://mlsanity.com/paper/2211.04867/) (2022)
- [Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker](https://mlsanity.com/paper/2310.10248/) (2023)
- [Multi-IMU with Online Self-Consistency for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2306.16197/) (2023)
- [Fine-grained Context and Multi-modal Alignment for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2407.04242/) (2024)
- [TUS-REC2024: A Challenge to Reconstruct 3D Freehand Ultrasound Without External Tracker](https://mlsanity.com/paper/2506.21765/) (2025)
- [Nonrigid Reconstruction of Freehand Ultrasound without a Tracker](https://mlsanity.com/paper/2407.05767/) (2024)
- [MoNetV2: Enhanced Motion Network for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2506.15835/) (2025)
- [Breast Lesion Diagnosis Using Static Images and Dynamic Video](https://mlsanity.com/paper/2308.09980/) (2023)
- [Enhancing Free-hand 3D Photoacoustic and Ultrasound Reconstruction using Deep Learning](https://mlsanity.com/paper/2502.03505/) (2025)
- [DualTrack: Sensorless 3D Ultrasound needs Local and Global Context](https://mlsanity.com/paper/2509.09530/) (2025)
- [Privileged Anatomical and Protocol Discrimination in Trackerless 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2308.10293/) (2023)
- [UltrasODM: A Dual Stream Optical Flow Mamba Network for 3D Freehand Ultrasound Reconstruction](https://mlsanity.com/paper/2512.07756/) (2025)

## Abstract (arXiv)

3D ultrasound (US) is widely used for its rich diagnostic information. However, it is criticized for its limited field of view. 3D freehand US reconstruction is promising in addressing the problem by providing broad range and freeform scan. The existing deep learning based methods only focus on the basic cases of skill sequences, and the model relies on the training data heavily. The sequences in real clinical practice are a mix of diverse skills and have complex scanning paths. Besides, deep models should adapt themselves to the testing cases with prior knowledge for better robustness, rather than only fit to the training cases. In this paper, we propose a novel approach to sensorless freehand 3D US reconstruction considering the complex skill sequences. Our contribution is three-fold. First, we advance a novel online learning framework by designing a differentiable reconstruction algorithm. It realizes an end-to-end optimization from section sequences to the reconstructed volume. Second, a self-supervised learning method is developed to explore the context information that reconstructed by the testing data itself, promoting the perception of the model. Third, inspired by the effectiveness of shape prior, we also introduce adversarial training to strengthen the learning of anatomical shape prior in the reconstructed volume. By mining the context and structural cues of the testing data, our online learning methods can drive the model to handle complex skill sequences. Experimental results on developmental dysplasia of the hip US and fetal US datasets show that, our proposed method can outperform the start-of-the-art methods regarding the shift errors and path similarities.