# Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing

arXiv: https://arxiv.org/abs/2307.11638 · HTML: https://mlsanity.com/paper/2307.11638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 16 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 6 of 6)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Learning to Autofocus](https://mlsanity.com/paper/2004.12260/) (2020)
- [Rapid and robust endoscopic content area estimation: A lean GPU-based pipeline and curated benchmark dataset](https://mlsanity.com/paper/2210.14771/) (2022)
- [A Robotic Auto-Focus System based on Deep Reinforcement Learning](https://mlsanity.com/paper/1809.03314/) (2018)
- [DASHA: Decentralized Autofocusing System with Hierarchical Agents](https://mlsanity.com/paper/2108.12842/) (2021)

## Abstract (arXiv)

Hyperspectral imaging (HSI) captures a greater level of spectral detail than traditional optical imaging, making it a potentially valuable intraoperative tool when precise tissue differentiation is essential. Hardware limitations of current optical systems used for handheld real-time video HSI result in a limited focal depth, thereby posing usability issues for integration of the technology into the operating room. This work integrates a focus-tunable liquid lens into a video HSI exoscope, and proposes novel video autofocusing methods based on deep reinforcement learning. A first-of-its-kind robotic focal-time scan was performed to create a realistic and reproducible testing dataset. We benchmarked our proposed autofocus algorithm against traditional policies, and found our novel approach to perform significantly (p<0.05) better than traditional techniques (0.070±.098 mean absolute focal error compared to 0.146±.148). In addition, we performed a blinded usability trial by having two neurosurgeons compare the system with different autofocus policies, and found our novel approach to be the most favourable, making our system a desirable addition for intraoperative HSI.