# Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis

arXiv: https://arxiv.org/abs/2203.10977 · HTML: https://mlsanity.com/paper/2203.10977/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [The Importance of Skip Connections in Biomedical Image Segmentation](https://mlsanity.com/paper/1608.04117/) (2016)
- [Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation](https://mlsanity.com/paper/1705.08302/) (2017)
- [Fast Interactive Object Annotation with Curve-GCN](https://mlsanity.com/paper/1903.06874/) (2019)
- [Common Limitations of Image Processing Metrics: A Picture Story](https://mlsanity.com/paper/2104.05642/) (2021)
- [Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation](https://mlsanity.com/paper/2106.12387/) (2021)
- [Voxel2Mesh: 3D Mesh Model Generation from Volumetric Data](https://mlsanity.com/paper/1912.03681/) (2019)
- [DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images](https://mlsanity.com/paper/1810.00111/) (2018)
- [High-level Prior-based Loss Functions for Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2011.08018/) (2020)
- [DeepSSM: A Blueprint for Image-to-Shape Deep Learning Models](https://mlsanity.com/paper/2110.07152/) (2021)
- [A survey on shape-constraint deep learning for medical image segmentation](https://mlsanity.com/paper/2101.07721/) (2021)
- [Anatomical Priors for Image Segmentation via Post-Processing with Denoising Autoencoders](https://mlsanity.com/paper/1906.02343/) (2019)
- [Sub-cortical brain structure segmentation using F-CNN's](https://mlsanity.com/paper/1602.02130/) (2016)
- [Hybrid graph convolutional neural networks for landmark-based anatomical segmentation](https://mlsanity.com/paper/2106.09832/) (2021)

## Built on by (most cited first, 25 of 25)

- [MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine](https://mlsanity.com/paper/2408.02900/) (2024)
- [CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images](https://mlsanity.com/paper/2307.03293/) (2023)
- [Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh Reconstruction in Cardiovascular MRI](https://mlsanity.com/paper/2311.13706/) (2023)
- [Multi-center anatomical segmentation with heterogeneous labels via landmark-based models](https://mlsanity.com/paper/2211.07395/) (2022)
- [Unsupervised Domain Adaptation for Anatomical Landmark Detection](https://mlsanity.com/paper/2308.13286/) (2023)
- [Instruction-Guided Lesion Segmentation for Chest X-rays with Automatically Generated Large-Scale Dataset](https://mlsanity.com/paper/2511.15186/) (2025)
- [Towards Robust Cardiac Segmentation using Graph Convolutional Networks](https://mlsanity.com/paper/2310.01210/) (2023)
- [The Past, Current, and Future of Neonatal Intensive Care Units with Artificial Intelligence](https://mlsanity.com/paper/2302.00225/) (2023)
- [Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training](https://mlsanity.com/paper/2401.08469/) (2024)
- [CheXmask-U: Quantifying uncertainty in landmark-based anatomical segmentation for X-ray images](https://mlsanity.com/paper/2512.10715/) (2025)
- [Mask-Guided Attention Regulation for Anatomically Consistent Counterfactual CXR Synthesis](https://mlsanity.com/paper/2603.04130/) (2026)
- [Unsupervised bias discovery in medical image segmentation](https://mlsanity.com/paper/2309.00451/) (2023)
- [Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks](https://mlsanity.com/paper/2311.11782/) (2023)
- [DenseSeg: Joint Learning for Semantic Segmentation and Landmark Detection Using Dense Image-to-Shape Representation](https://mlsanity.com/paper/2405.19746/) (2024)
- [Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling](https://mlsanity.com/paper/2502.12713/) (2025)
- [Revisiting Data Scaling in Medical Image Segmentation via Topology-Aware Augmentation](https://mlsanity.com/paper/2511.13883/) (2025)
- [Mask-HybridGNet: Graph-based segmentation with emergent anatomical correspondence from pixel-level supervision](https://mlsanity.com/paper/2602.21179/) (2026)
- [Joint Dense-Point Representation for Contour-Aware Graph Segmentation](https://mlsanity.com/paper/2306.12155/) (2023)
- [Fitting Skeletal Models via Graph-based Learning](https://mlsanity.com/paper/2409.05311/) (2024)
- [Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain](https://mlsanity.com/paper/2411.16123/) (2024)
- [Divide and Conquer Self-Supervised Learning for High-Content Imaging](https://mlsanity.com/paper/2503.07444/) (2025)
- [Interpretable graph-based models on multimodal biomedical data integration: A technical review and benchmarking](https://mlsanity.com/paper/2505.01696/) (2025)
- [Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging](https://mlsanity.com/paper/2510.05971/) (2025)
- [Noise Injection: Improving Out-of-Distribution Generalization for Limited Size Datasets](https://mlsanity.com/paper/2511.03855/) (2025)
- [Self-Supervised Temporal Regularization for Landmark-Based Cardiac Segmentation with Automatic AHA Regional Mapping](https://mlsanity.com/paper/2606.31785/) (2026)

## Abstract (arXiv)

Anatomical segmentation is a fundamental task in medical image computing, generally tackled with fully convolutional neural networks which produce dense segmentation masks. These models are often trained with loss functions such as cross-entropy or Dice, which assume pixels to be independent of each other, thus ignoring topological errors and anatomical inconsistencies. We address this limitation by moving from pixel-level to graph representations, which allow to naturally incorporate anatomical constraints by construction. To this end, we introduce HybridGNet, an encoder-decoder neural architecture that leverages standard convolutions for image feature encoding and graph convolutional neural networks (GCNNs) to decode plausible representations of anatomical structures. We also propose a novel image-to-graph skip connection layer which allows localized features to flow from standard convolutional blocks to GCNN blocks, and show that it improves segmentation accuracy. The proposed architecture is extensively evaluated in a variety of domain shift and image occlusion scenarios, and audited considering different types of demographic domain shift. Our comprehensive experimental setup compares HybridGNet with other landmark and pixel-based models for anatomical segmentation in chest x-ray images, and shows that it produces anatomically plausible results in challenging scenarios where other models tend to fail.