# Exploring the robustness of TractOracle methods in RL-based tractography

arXiv: https://arxiv.org/abs/2507.11486 · HTML: https://mlsanity.com/paper/2507.11486/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 64 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Randomized Ensembled Double Q-Learning: Learning Fast Without a Model](https://mlsanity.com/paper/2101.05982/) (2021)
- [Defining and Characterizing Reward Hacking](https://mlsanity.com/paper/2209.13085/) (2022)
- [Dropout Q-Functions for Doubly Efficient Reinforcement Learning](https://mlsanity.com/paper/2110.02034/) (2021)
- [CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity](https://mlsanity.com/paper/1902.05605/) (2019)
- [TractSeg - Fast and accurate white matter tract segmentation](https://mlsanity.com/paper/1805.07103/) (2018)
- [Training Larger Networks for Deep Reinforcement Learning](https://mlsanity.com/paper/2102.07920/) (2021)
- [Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?](https://mlsanity.com/paper/2003.01629/) (2020)
- [Filtering in tractography using autoencoders (FINTA)](https://mlsanity.com/paper/2010.04007/) (2020)
- [Tractography and machine learning: Current state and open challenges](https://mlsanity.com/paper/1902.05568/) (2019)
- [TractOracle: towards an anatomically-informed reward function for RL-based tractography](https://mlsanity.com/paper/2403.17845/) (2024)
- [What Matters in Reinforcement Learning for Tractography](https://mlsanity.com/paper/2305.09041/) (2023)
- [Supervised Tractogram Filtering using Geometric Deep Learning](https://mlsanity.com/paper/2212.03300/) (2022)

## Built on by (most cited first, 1 of 1)

- [TractRLFusion: A GPT-Based Multi-Critic Policy Fusion Framework for Fiber Tractography](https://mlsanity.com/paper/2601.13897/) (2026)

## Abstract (arXiv)

Tractography algorithms leverage diffusion MRI to reconstruct the fibrous architecture of the brain's white matter. Among machine learning approaches, reinforcement learning (RL) has emerged as a promising framework for tractography, outperforming traditional methods in several key aspects. TractOracle-RL, a recent RL-based approach, reduces false positives by incorporating anatomical priors into the training process via a reward-based mechanism. In this paper, we investigate four extensions of the original TractOracle-RL framework by integrating recent advances in RL, and we evaluate their performance across five diverse diffusion MRI datasets. Results demonstrate that combining an oracle with the RL framework consistently leads to robust and reliable tractography, regardless of the specific method or dataset used. We also introduce a novel RL training scheme called Iterative Reward Training (IRT), inspired by the Reinforcement Learning from Human Feedback (RLHF) paradigm. Instead of relying on human input, IRT leverages bundle filtering methods to iteratively refine the oracle's guidance throughout training. Experimental results show that RL methods trained with oracle feedback significantly outperform widely used tractography techniques in terms of accuracy and anatomical validity.