# A Multimodal Deep Learning Approach for White Matter Shape Prediction in Diffusion MRI Tractography

arXiv: https://arxiv.org/abs/2504.18400 · HTML: https://mlsanity.com/paper/2504.18400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 86 references that are in the map.
- First posted on arXiv 25 Apr 2025.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data](https://mlsanity.com/paper/2303.14080/) (2023)
- [Superficial White Matter Analysis: An Efficient Point-cloud-based Deep Learning Framework with Supervised Contrastive Learning for Consistent Tractography Parcellation across Populations and dMRI Acquisitions](https://mlsanity.com/paper/2207.08975/) (2022)
- [Tractogram filtering of anatomically non-plausible fibers with geometric deep learning](https://mlsanity.com/paper/2003.11013/) (2020)
- [SuperDTI: Ultrafast diffusion tensor imaging and fiber tractography with deep learning](https://mlsanity.com/paper/2002.01031/) (2020)
- [Deep fiber clustering: Anatomically informed fiber clustering with self-supervised deep learning for fast and effective tractography parcellation](https://mlsanity.com/paper/2205.00627/) (2022)
- [TractGeoNet: A geometric deep learning framework for pointwise analysis of tract microstructure to predict language assessment performance](https://mlsanity.com/paper/2307.03982/) (2023)
- [TractGraphCNN: anatomically informed graph CNN for classification using diffusion MRI tractography](https://mlsanity.com/paper/2301.01911/) (2023)
- [TractoSCR: A Novel Supervised Contrastive Regression Framework for Prediction of Neurocognitive Measures Using Multi-Site Harmonized Diffusion MRI Tractography](https://mlsanity.com/paper/2210.07411/) (2022)
- [MM-GTUNets: Unified Multi-Modal Graph Deep Learning for Brain Disorders Prediction](https://mlsanity.com/paper/2406.14455/) (2024)
- [PPA: Principal Parcellation Analysis for Brain Connectomes and Multiple Traits](https://mlsanity.com/paper/2103.03478/) (2021)
- [Cross-domain Fiber Cluster Shape Analysis for Language Performance Cognitive Score Prediction](https://mlsanity.com/paper/2403.19001/) (2024)
- [The shape of the brain's connections is predictive of cognitive performance: an explainable machine learning study](https://mlsanity.com/paper/2410.15108/) (2024)
- [A Novel Deep Learning Tractography Fiber Clustering Framework for Functionally Consistent White Matter Parcellation Using Multimodal Diffusion MRI and Functional MRI](https://mlsanity.com/paper/2411.01859/) (2024)
- [TractCloud-FOV: Deep Learning-based Robust Tractography Parcellation in Diffusion MRI with Incomplete Field of View](https://mlsanity.com/paper/2502.20637/) (2025)
- [White Matter Geometry-Guided Score-Based Diffusion Model for Tissue Microstructure Imputation in Tractography Imaging](https://mlsanity.com/paper/2407.19460/) (2024)
- [TractShapeNet: Efficient Multi-Shape Learning with 3D Tractography Point Clouds](https://mlsanity.com/paper/2410.22099/) (2024)

## Abstract (arXiv)

Shape measures have emerged as promising descriptors of white matter tractography, offering complementary insights into anatomical variability and associations with cognitive and clinical phenotypes. However, conventional methods for computing shape measures are computationally expensive and time-consuming for large-scale datasets due to reliance on voxel-based representations. We propose Tract2Shape, a novel multimodal deep learning framework that leverages geometric (point cloud) and scalar (tabular) features to predict ten white matter tractography shape measures. To enhance model efficiency, we utilize a dimensionality reduction algorithm for the model to predict five primary shape components. The model is trained and evaluated on two independently acquired datasets, the HCP-YA dataset, and the PPMI dataset. We evaluate the performance of Tract2Shape by training and testing it on the HCP-YA dataset and comparing the results with state-of-the-art models. To further assess its robustness and generalization ability, we also test Tract2Shape on the unseen PPMI dataset. Tract2Shape outperforms SOTA deep learning models across all ten shape measures, achieving the highest average Pearson's r and the lowest nMSE on the HCP-YA dataset. The ablation study shows that both multimodal input and PCA contribute to performance gains. On the unseen testing PPMI dataset, Tract2Shape maintains a high Pearson's r and low nMSE, demonstrating strong generalizability in cross-dataset evaluation. Tract2Shape enables fast, accurate, and generalizable prediction of white matter shape measures from tractography data, supporting scalable analysis across datasets. This framework lays a promising foundation for future large-scale white matter shape analysis.