# Benefits of Linear Conditioning with Metadata for Image Segmentation

arXiv: https://arxiv.org/abs/2102.09582 · HTML: https://mlsanity.com/paper/2102.09582/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [Modulating early visual processing by language](https://mlsanity.com/paper/1707.00683/) (2017)
- [Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation](https://mlsanity.com/paper/1904.06346/) (2019)
- [Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition](https://mlsanity.com/paper/1707.06065/) (2017)
- [ivadomed: A Medical Imaging Deep Learning Toolbox](https://mlsanity.com/paper/2010.09984/) (2020)
- [Annotation-cost Minimization for Medical Image Segmentation using Suggestive Mixed Supervision Fully Convolutional Networks](https://mlsanity.com/paper/1812.11302/) (2018)
- [Learning to segment images with classification labels](https://mlsanity.com/paper/1912.12533/) (2019)
- [Automatic segmentation of spinal multiple sclerosis lesions: How to generalize across MRI contrasts?](https://mlsanity.com/paper/2003.04377/) (2020)
- [Multiclass Spinal Cord Tumor Segmentation on MRI with Deep Learning](https://mlsanity.com/paper/2012.12820/) (2020)

## Built on by (most cited first, 6 of 6)

- [Label fusion and training methods for reliable representation of inter-rater uncertainty](https://mlsanity.com/paper/2202.07550/) (2022)
- [Impact of individual rater style on deep learning uncertainty in medical imaging segmentation](https://mlsanity.com/paper/2105.02197/) (2021)
- [Metadata Improves Segmentation Through Multitasking Elicitation](https://mlsanity.com/paper/2308.09411/) (2023)
- [Gaussian Random Fields as an Abstract Representation of Patient Metadata for Multimodal Medical Image Segmentation](https://mlsanity.com/paper/2503.05214/) (2025)
- [Spatial Feature-wise Linear Modulation (SpFiLM) for Contrast Agent-Aware Brain Parcellation](https://mlsanity.com/paper/2609.07718/) (2026)
- [SV-Cine: Diagnosis-Conditioned Segmentation of Single Ventricle Physiology via Generative Data Augmentation](https://mlsanity.com/paper/2609.12997/) (2026)

## Abstract (arXiv)

Medical images are often accompanied by metadata describing the image (vendor, acquisition parameters) and the patient (disease type or severity, demographics, genomics). This metadata is usually disregarded by image segmentation methods. In this work, we adapt a linear conditioning method called FiLM (Feature-wise Linear Modulation) for image segmentation tasks. This FiLM adaptation enables integrating metadata into segmentation models for better performance. We observed an average Dice score increase of 5.1% on spinal cord tumor segmentation when incorporating the tumor type with FiLM. The metadata modulates the segmentation process through low-cost affine transformations applied on feature maps which can be included in any neural network's architecture. Additionally, we assess the relevance of segmentation FiLM layers for tackling common challenges in medical imaging: multi-class training with missing segmentations, model adaptation to multiple tasks, and training with a limited or unbalanced number of annotated data. Our results demonstrated the following benefits of FiLM for segmentation: FiLMed U-Net was robust to missing labels and reached higher Dice scores with few labels (up to 16.7%) compared to single-task U-Net. The code is open-source and available at www.ivadomed.org.