# Sterilizable Scene Graph Generation for Operating Rooms

arXiv: https://arxiv.org/abs/2608.16469 · HTML: https://mlsanity.com/paper/2608.16469/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Neural Motifs: Scene Graph Parsing with Global Context](https://mlsanity.com/paper/1711.06640/) (2017)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets](https://mlsanity.com/paper/2007.05405/) (2020)
- [Segmentation-grounded Scene Graph Generation](https://mlsanity.com/paper/2104.14207/) (2021)
- [Med-NCA: Robust and Lightweight Segmentation with Neural Cellular Automata](https://mlsanity.com/paper/2302.03473/) (2023)
- [ORacle: Large Vision-Language Models for Knowledge-Guided Holistic OR Domain Modeling](https://mlsanity.com/paper/2404.07031/) (2024)
- [M3D-NCA: Robust 3D Segmentation with Built-in Quality Control](https://mlsanity.com/paper/2309.02954/) (2023)
- [Unsupervised Training of Neural Cellular Automata on Edge Devices](https://mlsanity.com/paper/2407.18114/) (2024)
- [Voxel Scene Graph for Intracranial Hemorrhage](https://mlsanity.com/paper/2407.21580/) (2024)
- [SASVi -- Segment Any Surgical Video](https://mlsanity.com/paper/2502.09653/) (2025)
- [CAT-SG: A Large Dynamic Scene Graph Dataset for Fine-Grained Understanding of Cataract Surgery](https://mlsanity.com/paper/2506.21813/) (2025)
- [OctreeNCA: Single-Pass 184 MP Segmentation on Consumer Hardware](https://mlsanity.com/paper/2508.06993/) (2025)
- [Equitable Federated Learning with NCA](https://mlsanity.com/paper/2506.21735/) (2025)
- [Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgery](https://mlsanity.com/paper/2509.20941/) (2025)

## Built on by (most cited first, 1 of 1)

- [Applications of Neural Cellular Automata: State of the Art, Challenges and Opportunities](https://mlsanity.com/paper/2609.24595/) (2026)

## Abstract (arXiv)

Scene graph generation from surgical video enables a holistic and structured understanding of surgical scenes by modeling objects and their semantic relationships. Despite recent advances, state-of-the-art approaches rely on large, parameter-heavy deep learning models that are impractical for deployment in the operating room (OR) due to hardware footprint, hygiene constraints, latency, and data privacy concerns. To the best of our knowledge, this is the first scene graph generation method built on NCAs and the first NCA framework capable of learning structured representations. We introduce SG-NCA, a lightweight scene graph generation framework based on Neural Cellular Automata (NCA), designed for inference in fanless devices critical for OR hygiene protocols. SG-NCA is the first scene graph generation combining NCA-based multiclass segmentation for efficient object detection and feature extraction with a lightweight relation predictor. We evaluate SG-NCA on videos of cataract surgery and cholecystectomy, demonstrating performance comparable to established baselines while requiring 55x fewer parameters. We showcase deployment on fanless edge devices better suited for the OR and demonstrate downstream applications such as surgical video captioning, highlighting SG-NCA's potential for affordable, privacy-preserving, and OR-ready intraoperative scene understanding.