# An MRF-UNet Product of Experts for Image Segmentation

arXiv: https://arxiv.org/abs/2104.05495 · HTML: https://mlsanity.com/paper/2104.05495/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [Unsupervised domain adaptation in brain lesion segmentation with adversarial networks](https://mlsanity.com/paper/1612.08894/) (2016)
- [A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols](https://mlsanity.com/paper/1805.10170/) (2018)
- [A Learning Strategy for Contrast-agnostic MRI Segmentation](https://mlsanity.com/paper/2003.01995/) (2020)
- [Learning Deep Structured Models](https://mlsanity.com/paper/1407.2538/) (2014)
- [3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study](https://mlsanity.com/paper/1612.03925/) (2016)
- [Partial Volume Segmentation of Brain MRI Scans of any Resolution and Contrast](https://mlsanity.com/paper/2004.10221/) (2020)
- [A Regularized Convolutional Neural Network for Semantic Image Segmentation](https://mlsanity.com/paper/1907.05287/) (2019)
- [PSACNN: Pulse Sequence Adaptive Fast Whole Brain Segmentation](https://mlsanity.com/paper/1901.05992/) (2019)
- [Conditional Random Fields as Recurrent Neural Networks for 3D Medical Imaging Segmentation](https://mlsanity.com/paper/1807.07464/) (2018)
- [Nonlinear Markov Random Fields Learned via Backpropagation](https://mlsanity.com/paper/1902.10747/) (2019)

## Built on by (most cited first, 2 of 2)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [SceneProp: Combining Neural Network and Markov Random Field for Scene-Graph Grounding](https://mlsanity.com/paper/2512.00936/) (2025)

## Abstract (arXiv)

While convolutional neural networks (CNNs) trained by back-propagation have seen unprecedented success at semantic segmentation tasks, they are known to struggle on out-of-distribution data. Markov random fields (MRFs) on the other hand, encode simpler distributions over labels that, although less flexible than UNets, are less prone to over-fitting. In this paper, we propose to fuse both strategies by computing the product of distributions of a UNet and an MRF. As this product is intractable, we solve for an approximate distribution using an iterative mean-field approach. The resulting MRF-UNet is trained jointly by back-propagation. Compared to other works using conditional random fields (CRFs), the MRF has no dependency on the imaging data, which should allow for less over-fitting. We show on 3D neuroimaging data that this novel network improves generalisation to out-of-distribution samples. Furthermore, it allows the overall number of parameters to be reduced while preserving high accuracy. These results suggest that a classic MRF smoothness prior can allow for less over-fitting when principally integrated into a CNN model. Our implementation is available at https://github.com/balbasty/nitorch.