# Three-dimensional multimodal medical imaging system based on free-hand ultrasound and structured light

arXiv: https://arxiv.org/abs/2105.14355 · HTML: https://mlsanity.com/paper/2105.14355/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 52 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 4 of 4)

- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Deep ChArUco: Dark ChArUco Marker Pose Estimation](https://mlsanity.com/paper/1812.03247/) (2018)
- [Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks](https://mlsanity.com/paper/1707.05392/) (2017)
- [MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation](https://mlsanity.com/paper/2105.00368/) (2021)

## Abstract (arXiv)

We propose a three-dimensional (3D) multimodal medical imaging system that combines freehand ultrasound and structured light 3D reconstruction in a single coordinate system without requiring registration. To the best of our knowledge, these techniques have not been combined before as a multimodal imaging technique. The system complements the internal 3D information acquired with ultrasound, with the external surface measured with the structure light technique. Moreover, the ultrasound probe's optical tracking for pose estimation was implemented based on a convolutional neural network. Experimental results show the system's high accuracy and reproducibility, as well as its potential for preoperative and intraoperative applications. The experimental multimodal error, or the distance from two surfaces obtained with different modalities, was 0.12 mm. The code is available as a Github repository.