# m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2008.10134 · HTML: https://mlsanity.com/paper/2008.10134/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2020.

## Built on by (most cited first, 26 of 26)

- [GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI](https://mlsanity.com/paper/2408.03361/) (2024)
- [LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching](https://mlsanity.com/paper/2306.11925/) (2023)
- [CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection](https://mlsanity.com/paper/2302.06294/) (2023)
- [Holistic Evaluation of GPT-4V for Biomedical Imaging](https://mlsanity.com/paper/2312.05256/) (2023)
- [Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentation](https://mlsanity.com/paper/2508.20909/) (2025)
- [Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports](https://mlsanity.com/paper/2407.05758/) (2024)
- [MedicoSAM: Robust Improvement of SAM for Medical Imaging](https://mlsanity.com/paper/2501.11734/) (2025)
- [Action Recognition in Video Recordings from Gynecologic Laparoscopy](https://mlsanity.com/paper/2311.18666/) (2023)
- [Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies](https://mlsanity.com/paper/2312.04344/) (2023)
- [Phase-Informed Tool Segmentation for Manual Small-Incision Cataract Surgery](https://mlsanity.com/paper/2411.16794/) (2024)
- [Deep learning approaches to surgical video segmentation and object detection: A Scoping Review](https://mlsanity.com/paper/2502.16459/) (2025)
- [MARL-MambaContour: Unleashing Multi-Agent Deep Reinforcement Learning for Active Contour Optimization in Medical Image Segmentation](https://mlsanity.com/paper/2506.18679/) (2025)
- [Is Visual in-Context Learning for Compositional Medical Tasks within Reach?](https://mlsanity.com/paper/2507.00868/) (2025)
- [MedicalNarratives: Connecting Medical Vision and Language with Localized Narratives](https://mlsanity.com/paper/2501.04184/) (2025)
- [SurgiSAM2: Fine-tuning a foundational model for surgical video anatomy segmentation and detection](https://mlsanity.com/paper/2503.03942/) (2025)
- [Project Imaging-X: A Survey of 1000+ Open-Access Medical Imaging Datasets for Foundation Model Development](https://mlsanity.com/paper/2603.27460/) (2026)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning](https://mlsanity.com/paper/2402.17903/) (2024)
- [Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine](https://mlsanity.com/paper/2411.19447/) (2024)
- [Exploring Compositional Generalization of Multimodal LLMs for Medical Imaging](https://mlsanity.com/paper/2412.20070/) (2024)
- [LapFM: A Laparoscopic Segmentation Foundation Model via Hierarchical Concept Evolving Pre-training](https://mlsanity.com/paper/2512.08439/) (2025)
- [Cholec80-port: A Geometrically Consistent Trocar Port Segmentation Dataset for Robust Surgical Scene Understanding](https://mlsanity.com/paper/2602.17060/) (2026)
- [Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework](https://mlsanity.com/paper/2604.02877/) (2026)
- [LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes](https://mlsanity.com/paper/2207.07418/) (2022)
- [Probing Intrinsic Medical Task Relationships: A Contrastive Learning Perspective](https://mlsanity.com/paper/2604.05651/) (2026)
- [Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction](https://mlsanity.com/paper/2607.14968/) (2026)

## Abstract (arXiv)

Autonomous surgical procedures, in particular minimal invasive surgeries, are the next frontier for Artificial Intelligence research. However, the existing challenges include precise identification of the human anatomy and the surgical settings, and modeling the environment for training of an autonomous agent. To address the identification of human anatomy and the surgical settings, we propose a deep learning based semantic segmentation algorithm to identify and label the tissues and organs in the endoscopic video feed of the human torso region. We present an annotated dataset, m2caiSeg, created from endoscopic video feeds of real-world surgical procedures. Overall, the data consists of 307 images, each of which is annotated for the organs and different surgical instruments present in the scene. We propose and train a deep convolutional neural network for the semantic segmentation task. To cater for the low quantity of annotated data, we use unsupervised pre-training and data augmentation. The trained model is evaluated on an independent test set of the proposed dataset. We obtained a F1 score of 0.33 while using all the labeled categories for the semantic segmentation task. Secondly, we labeled all instruments into an 'Instruments' superclass to evaluate the model's performance on discerning the various organs and obtained a F1 score of 0.57. We propose a new dataset and a deep learning method for pixel level identification of various organs and instruments in a endoscopic surgical scene. Surgical scene understanding is one of the first steps towards automating surgical procedures.