# TractoTransformer: Diffusion MRI Streamline Tractography using CNN and Transformer Networks

arXiv: https://arxiv.org/abs/2509.16429 · HTML: https://mlsanity.com/paper/2509.16429/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SuperDTI: Ultrafast diffusion tensor imaging and fiber tractography with deep learning](https://mlsanity.com/paper/2002.01031/) (2020)
- [DeepTract: A Probabilistic Deep Learning Framework for White Matter Fiber Tractography](https://mlsanity.com/paper/1812.05129/) (2018)
- [Tractography and machine learning: Current state and open challenges](https://mlsanity.com/paper/1902.05568/) (2019)
- [Tract orientation mapping for bundle-specific tractography](https://mlsanity.com/paper/1806.05580/) (2018)
- [HAMLET: Hierarchical Harmonic Filters for Learning Tracts from Diffusion MRI](https://mlsanity.com/paper/1807.01068/) (2018)

## Built on by (most cited first, 1 of 1)

- [A foundation for systematic analysis of transformers and RNNs for tractography](https://mlsanity.com/paper/2610.01894/) (2026)

## Abstract (arXiv)

White matter tractography is an advanced neuroimaging technique that reconstructs the 3D white matter pathways of the brain from diffusion MRI data. It can be framed as a pathfinding problem aiming to infer neural fiber trajectories from noisy and ambiguous measurements, facing challenges such as crossing, merging, and fanning white-matter configurations. In this paper, we propose a novel tractography method that leverages Transformers to model the sequential nature of white matter streamlines, enabling the prediction of fiber directions by integrating both the trajectory context and current diffusion MRI measurements. To incorporate spatial information, we utilize CNNs that extract microstructural features from local neighborhoods around each voxel. By combining these complementary sources of information, our approach improves the precision and completeness of neural pathway mapping compared to traditional tractography models. We evaluate our method with the Tractometer toolkit, achieving competitive performance against state-of-the-art approaches, and present qualitative results on the TractoInferno dataset, demonstrating strong generalization to real-world data.