# DAISI: Database for AI Surgical Instruction

arXiv: https://arxiv.org/abs/2004.02809 · HTML: https://mlsanity.com/paper/2004.02809/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2020.

## Built on by (most cited first, 9 of 9)

- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures](https://mlsanity.com/paper/2307.15220/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Retrieval of surgical phase transitions using reinforcement learning](https://mlsanity.com/paper/2208.00902/) (2022)
- [Surgical Instruction Generation with Transformers](https://mlsanity.com/paper/2107.06964/) (2021)
- [Computational Teaching for Driving via Multi-Task Imitation Learning](https://mlsanity.com/paper/2410.01608/) (2024)
- [Rethinking Surgical Captioning: End-to-End Window-Based MLP Transformer Using Patches](https://mlsanity.com/paper/2207.00113/) (2022)
- [SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentary](https://mlsanity.com/paper/2605.21132/) (2026)
- [Spatio-Temporal Retrieval-based Priors for Adaptive Computational Teaching in Driving](https://mlsanity.com/paper/2606.25224/) (2026)

## Abstract (arXiv)

Telementoring surgeons as they perform surgery can be essential in the treatment of patients when in situ expertise is not available. Nonetheless, expert mentors are often unavailable to provide trainees with real-time medical guidance. When mentors are unavailable, a fallback autonomous mechanism should provide medical practitioners with the required guidance. However, AI/autonomous mentoring in medicine has been limited by the availability of generalizable prediction models, and surgical procedures datasets to train those models with. This work presents the initial steps towards the development of an intelligent artificial system for autonomous medical mentoring. Specifically, we present the first Database for AI Surgical Instruction (DAISI). DAISI leverages on images and instructions to provide step-by-step demonstrations of how to perform procedures from various medical disciplines. The dataset was acquired from real surgical procedures and data from academic textbooks. We used DAISI to train an encoder-decoder neural network capable of predicting medical instructions given a current view of the surgery. Afterwards, the instructions predicted by the network were evaluated using cumulative BLEU scores and input from expert physicians. According to the BLEU scores, the predicted and ground truth instructions were as high as 67% similar. Additionally, expert physicians subjectively assessed the algorithm using Likert scale, and considered that the predicted descriptions were related to the images. This work provides a baseline for AI algorithms to assist in autonomous medical mentoring.