# Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction

arXiv: https://arxiv.org/abs/2105.01456 · HTML: https://mlsanity.com/paper/2105.01456/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 4 May 2021.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [Segmenting Transparent Objects in the Wild](https://mlsanity.com/paper/2003.13948/) (2020)
- [Detecting Visual Relationships with Deep Relational Networks](https://mlsanity.com/paper/1704.03114/) (2017)
- [The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches](https://mlsanity.com/paper/1803.01164/) (2018)
- [A Review of Modularization Techniques in Artificial Neural Networks](https://mlsanity.com/paper/1904.12770/) (2019)
- [Computer vision-based recognition of liquid surfaces and phase boundaries in transparent vessels, with emphasis on chemistry applications](https://mlsanity.com/paper/1404.7174/) (2014)
- [Identifying Spatial Relations in Images using Convolutional Neural Networks](https://mlsanity.com/paper/1706.04215/) (2017)
- [A Novel Global Spatial Attention Mechanism in Convolutional Neural Network for Medical Image Classification](https://mlsanity.com/paper/2007.15897/) (2020)
- [Setting an attention region for convolutional neural networks using region selective features, for recognition of materials within glass vessels](https://mlsanity.com/paper/1708.08711/) (2017)
- [Hierarchical semantic segmentation using modular convolutional neural networks](https://mlsanity.com/paper/1710.05126/) (2017)

## Built on by (most cited first, 5 of 5)

- [Predicting 3D shapes, masks, and properties of materials, liquids, and objects inside transparent containers, using the TransProteus CGI dataset](https://mlsanity.com/paper/2109.07577/) (2021)
- [Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data](https://mlsanity.com/paper/2403.03309/) (2024)
- [Fluid Viscosity Prediction Leveraging Computer Vision and Robot Interaction](https://mlsanity.com/paper/2308.02715/) (2023)
- [ViscNet: Vision-Based In-line Viscometry for Fluid Mixing Process](https://mlsanity.com/paper/2512.01268/) (2025)
- [Learning to Segment Liquids in Real-world Images](https://mlsanity.com/paper/2601.00940/) (2026)

## Abstract (arXiv)

This work explores the use of computer vision for image segmentation and classification of medical fluid samples in transparent containers (for example, tubes, syringes, infusion bags). Handling fluids such as infusion fluids, blood, and urine samples is a significant part of the work carried out in medical labs and hospitals. The ability to accurately identify and segment the liquids and the vessels that contain them from images can help in automating such processes. Modern computer vision typically involves training deep neural nets on large datasets of annotated images. This work presents a new dataset containing 1,300 annotated images of medical samples involving vessels containing liquids and solid material. The images are annotated with the type of liquid (e.g., blood, urine), the phase of the material (e.g., liquid, solid, foam, suspension), the type of vessel (e.g., syringe, tube, cup, infusion bottle/bag), and the properties of the vessel (transparent, opaque). In addition, vessel parts such as corks, labels, spikes, and valves are annotated. Relations and hierarchies between vessels and materials are also annotated, such as which vessel contains which material or which vessels are linked or contain each other. Three neural networks are trained on the dataset: One network learns to detect vessels, a second net detects the materials and parts inside each vessel, and a third net identifies relationships and connectivity between vessels.