# Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology

arXiv: https://arxiv.org/abs/2101.06775 · HTML: https://mlsanity.com/paper/2101.06775/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 17 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [Coherent Semantic Attention for Image Inpainting](https://mlsanity.com/paper/1905.12384/) (2019)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [Contextual-based Image Inpainting: Infer, Match, and Translate](https://mlsanity.com/paper/1711.08590/) (2017)
- [Learning rotation invariant convolutional filters for texture classification](https://mlsanity.com/paper/1604.06720/) (2016)
- [Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition](https://mlsanity.com/paper/1908.01174/) (2019)
- [Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart](https://mlsanity.com/paper/1803.08943/) (2018)
- [Exploiting bilateral symmetry in brain lesion segmentation](https://mlsanity.com/paper/1907.08196/) (2019)

## Built on by (most cited first, 10 of 10)

- [Adapting Off-the-Shelf Source Segmenter for Target Medical Image Segmentation](https://mlsanity.com/paper/2106.12497/) (2021)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis](https://mlsanity.com/paper/2106.12499/) (2021)
- [Inpainting Pathology in Lumbar Spine MRI with Latent Diffusion](https://mlsanity.com/paper/2406.02477/) (2024)
- [Tagged-MRI Sequence to Audio Synthesis via Self Residual Attention Guided Heterogeneous Translator](https://mlsanity.com/paper/2206.02284/) (2022)
- [Automated interpretation of congenital heart disease from multi-view echocardiograms](https://mlsanity.com/paper/2311.18788/) (2023)
- [Neuroverse3D: Developing In-Context Learning Universal Model for Neuroimaging in 3D](https://mlsanity.com/paper/2503.02410/) (2025)
- [Post-Processing Methods for Improving Accuracy in MRI Inpainting](https://mlsanity.com/paper/2510.15282/) (2025)
- [Automated anomaly-aware 3D segmentation of bones and cartilages in knee MR images from the Osteoarthritis Initiative](https://mlsanity.com/paper/2211.16696/) (2022)
- [Inpainting Normal Maps for Lightstage data](https://mlsanity.com/paper/2401.08099/) (2024)

## Abstract (arXiv)

Deformable registration of magnetic resonance images between patients with brain tumors and healthy subjects has been an important tool to specify tumor geometry through location alignment and facilitate pathological analysis. Since tumor region does not match with any ordinary brain tissue, it has been difficult to deformably register a patients brain to a normal one. Many patient images are associated with irregularly distributed lesions, resulting in further distortion of normal tissue structures and complicating registration's similarity measure. In this work, we follow a multi-step context-aware image inpainting framework to generate synthetic tissue intensities in the tumor region. The coarse image-to-image translation is applied to make a rough inference of the missing parts. Then, a feature-level patch-match refinement module is applied to refine the details by modeling the semantic relevance between patch-wise features. A symmetry constraint reflecting a large degree of anatomical symmetry in the brain is further proposed to achieve better structure understanding. Deformable registration is applied between inpainted patient images and normal brains, and the resulting deformation field is eventually used to deform original patient data for the final alignment. The method was applied to the Multimodal Brain Tumor Segmentation (BraTS) 2018 challenge database and compared against three existing inpainting methods. The proposed method yielded results with increased peak signal-to-noise ratio, structural similarity index, inception score, and reduced L1 error, leading to successful patient-to-normal brain image registration.