# Nonrigid Reconstruction of Freehand Ultrasound without a Tracker

arXiv: https://arxiv.org/abs/2407.05767 · HTML: https://mlsanity.com/paper/2407.05767/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks](https://mlsanity.com/paper/1707.05392/) (2017)
- [Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2108.00274/) (2021)
- [Deep Motion Network for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2207.00177/) (2022)
- [Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames](https://mlsanity.com/paper/2211.04867/) (2022)
- [Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker](https://mlsanity.com/paper/2310.10248/) (2023)
- [Multi-IMU with Online Self-Consistency for Freehand 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2306.16197/) (2023)
- [Privileged Anatomical and Protocol Discrimination in Trackerless 3D Ultrasound Reconstruction](https://mlsanity.com/paper/2308.10293/) (2023)

## Built on by (most cited first, 3 of 3)

- [TUS-REC2024: A Challenge to Reconstruct 3D Freehand Ultrasound Without External Tracker](https://mlsanity.com/paper/2506.21765/) (2025)
- [DualTrack: Sensorless 3D Ultrasound needs Local and Global Context](https://mlsanity.com/paper/2509.09530/) (2025)
- [UltrasODM: A Dual Stream Optical Flow Mamba Network for 3D Freehand Ultrasound Reconstruction](https://mlsanity.com/paper/2512.07756/) (2025)

## Abstract (arXiv)

Reconstructing 2D freehand Ultrasound (US) frames into 3D space without using a tracker has recently seen advances with deep learning. Predicting good frame-to-frame rigid transformations is often accepted as the learning objective, especially when the ground-truth labels from spatial tracking devices are inherently rigid transformations. Motivated by a) the observed nonrigid deformation due to soft tissue motion during scanning, and b) the highly sensitive prediction of rigid transformation, this study investigates the methods and their benefits in predicting nonrigid transformations for reconstructing 3D US. We propose a novel co-optimisation algorithm for simultaneously estimating rigid transformations among US frames, supervised by ground-truth from a tracker, and a nonrigid deformation, optimised by a regularised registration network. We show that these two objectives can be either optimised using meta-learning or combined by weighting. A fast scattered data interpolation is also developed for enabling frequent reconstruction and registration of non-parallel US frames, during training. With a new data set containing over 357,000 frames in 720 scans, acquired from 60 subjects, the experiments demonstrate that, due to an expanded thus easier-to-optimise solution space, the generalisation is improved with the added deformation estimation, with respect to the rigid ground-truth. The global pixel reconstruction error (assessing accumulative prediction) is lowered from 18.48 to 16.51 mm, compared with baseline rigid-transformation-predicting methods. Using manually identified landmarks, the proposed co-optimisation also shows potentials in compensating nonrigid tissue motion at inference, which is not measurable by tracker-provided ground-truth. The code and data used in this paper are made publicly available at https://github.com/QiLi111/NR-Rec-FUS.