# Cross Modality 3D Navigation Using Reinforcement Learning and Neural Style Transfer

arXiv: https://arxiv.org/abs/2111.03485 · HTML: https://mlsanity.com/paper/2111.03485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Builds on (most cited first, 7 of 7)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Multiple Landmark Detection using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1907.00318/) (2019)
- [Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1806.03228/) (2018)
- [t-Soft Update of Target Network for Deep Reinforcement Learning](https://mlsanity.com/paper/2008.10861/) (2020)

## Abstract (arXiv)

This paper presents the use of Multi-Agent Reinforcement Learning (MARL) to perform navigation in 3D anatomical volumes from medical imaging. We utilize Neural Style Transfer to create synthetic Computed Tomography (CT) agent gym environments and assess the generalization capabilities of our agents to clinical CT volumes. Our framework does not require any labelled clinical data and integrates easily with several image translation techniques, enabling cross modality applications. Further, we solely condition our agents on 2D slices, breaking grounds for 3D guidance in much more difficult imaging modalities, such as ultrasound imaging. This is an important step towards user guidance during the acquisition of standardised diagnostic view planes, improving diagnostic consistency and facilitating better case comparison.