# Surgical Triplet Recognition via Diffusion Model

arXiv: https://arxiv.org/abs/2406.13210 · HTML: https://mlsanity.com/paper/2406.13210/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021)
- [Surgical Data Science -- from Concepts toward Clinical Translation](https://mlsanity.com/paper/2011.02284/) (2020)
- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets](https://mlsanity.com/paper/2007.05405/) (2020)
- [Towards Unified Surgical Skill Assessment](https://mlsanity.com/paper/2106.01035/) (2021)
- [Dissecting Self-Supervised Learning Methods for Surgical Computer Vision](https://mlsanity.com/paper/2207.00449/) (2022)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition](https://mlsanity.com/paper/2211.16963/) (2022)
- [CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection](https://mlsanity.com/paper/2302.06294/) (2023)
- [Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactions](https://mlsanity.com/paper/2307.09548/) (2023)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Self-distillation for surgical action recognition](https://mlsanity.com/paper/2303.12915/) (2023)
- [Why Deep Surgical Models Fail?: Revisiting Surgical Action Triplet Recognition through the Lens of Robustness](https://mlsanity.com/paper/2209.08647/) (2022)

## Built on by (most cited first, 4 of 4)

- [CliPPER: Contextual Video-Language Pretraining on Long-form Intraoperative Surgical Procedures for Event Recognition](https://mlsanity.com/paper/2603.24539/) (2026)
- [TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Models](https://mlsanity.com/paper/2603.06999/) (2026)
- [Latent-Action-Guided Video-Language Feature Learning for Surgical Instrument-Tissue Interaction Recognition](https://mlsanity.com/paper/2607.19889/) (2026)
- [TRCoRSurg: Temporal-Relational Co-Reasoning for Surgical Video Triplet Recognition](https://mlsanity.com/paper/2608.04606/) (2026)

## Abstract (arXiv)

Surgical triplet recognition is an essential building block to enable next-generation context-aware operating rooms. The goal is to identify the combinations of instruments, verbs, and targets presented in surgical video frames. In this paper, we propose DiffTriplet, a new generative framework for surgical triplet recognition employing the diffusion model, which predicts surgical triplets via iterative denoising. To handle the challenge of triplet association, two unique designs are proposed in our diffusion framework, i.e., association learning and association guidance. During training, we optimize the model in the joint space of triplets and individual components to capture the dependencies among them. At inference, we integrate association constraints into each update of the iterative denoising process, which refines the triplet prediction using the information of individual components. Experiments on the CholecT45 and CholecT50 datasets show the superiority of the proposed method in achieving a new state-of-the-art performance for surgical triplet recognition. Our codes will be released.