# COMPASS: A Formal Framework and Aggregate Dataset for Generalized Surgical Procedure Modeling

arXiv: https://arxiv.org/abs/2209.06424 · HTML: https://mlsanity.com/paper/2209.06424/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2022.

## Built on by (most cited first, 6 of 6)

- [Towards Surgical Context Inference and Translation to Gestures](https://mlsanity.com/paper/2302.14237/) (2023)
- [Robotic Scene Segmentation with Memory Network for Runtime Surgical Context Inference](https://mlsanity.com/paper/2308.12789/) (2023)
- [Machine Learning-Based Automated Assessment of Intracorporeal Suturing in Laparoscopic Fundoplication](https://mlsanity.com/paper/2412.16195/) (2024)
- [Safe Uncertainty-Aware Learning of Robotic Suturing](https://mlsanity.com/paper/2505.16596/) (2025)
- [MiDAS: A Multimodal Data Acquisition System and Dataset for Robot-Assisted Minimally Invasive Surgery](https://mlsanity.com/paper/2602.12407/) (2026)
- [Real-Time Multimodal Activity-Aware Error Detection in Robot-Assisted Surgery](https://mlsanity.com/paper/2606.23593/) (2026)

## Abstract (arXiv)

Purpose: We propose a formal framework for the modeling and segmentation of minimally-invasive surgical tasks using a unified set of motion primitives (MPs) to enable more objective labeling and the aggregation of different datasets. Methods: We model dry-lab surgical tasks as finite state machines, representing how the execution of MPs as the basic surgical actions results in the change of surgical context, which characterizes the physical interactions among tools and objects in the surgical environment. We develop methods for labeling surgical context based on video data and for automatic translation of context to MP labels. We then use our framework to create the COntext and Motion Primitive Aggregate Surgical Set (COMPASS), including six dry-lab surgical tasks from three publicly-available datasets (JIGSAWS, DESK, and ROSMA), with kinematic and video data and context and MP labels. Results: Our context labeling method achieves near-perfect agreement between consensus labels from crowd-sourcing and expert surgeons. Segmentation of tasks to MPs results in the creation of the COMPASS dataset that nearly triples the amount of data for modeling and analysis and enables the generation of separate transcripts for the left and right tools. Conclusion: The proposed framework results in high quality labeling of surgical data based on context and fine-grained MPs. Modeling surgical tasks with MPs enables the aggregation of different datasets and the separate analysis of left and right hands for bimanual coordination assessment. Our formal framework and aggregate dataset can support the development of explainable and multi-granularity models for improved surgical process analysis, skill assessment, error detection, and autonomy.