OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality
Not on the map. Why?
Abstract (arXiv)
High-frequency surgical maneuvers in ophthalmology necessitate high-fidelity temporal modeling, yet characterizing long-range procedural dependencies remains computationally prohibitive for attention-based architectures. Existing models often require aggressive temporal downsampling, which compromises the detection of fine-grained action boundaries and instrument-tissue interactions. To address these scalability constraints, we present OphBiWSSD, a framework that reformulates surgical temporal action localization leveraging Bidirectional State Space Duality. By employing a weight-tied selective scan mechanism that incorporates both preceding and succeeding surgical contexts, our approach facilitates the global synthesis of non-causal temporal cues with linear complexity. This streamlined architecture is well-suited to capture the bidirectional dependencies present in ophthalmic workflows, effectively bridging the gap between local boundary precision and long-range procedural context without incurring the quadratic memory overhead of traditional Transformers. Extensive experiments on the OphNet benchmark demonstrate that OphBiWSSD achieves state-of-the-art temporal localization performance, with mean Average Precisions of 44.42% on phases and 43.08% on operations, surpassing the baselines by 6.80% and 6.66%, respectively. Empirical validation indicates that our approach ensures precise temporal localization and offers a computationally viable pathway for deploying surgical intelligence systems in clinical environments. The code is publicly available at https://github.com/yo3nglau/OphBiWSSD.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
34 of 38 references.
References · 34
- 2017Attention Is All You Needcited by 59,932
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2018SlowFast Networks for Video Recognitioncited by 1,862
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
- 2021Video Swin Transformercited by 1,090
- 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
- 2016Deep Networks with Stochastic Depthcited by 973
- 2024VMamba: Visual State Space Modelcited by 955
- 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
22 more
- 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
- 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 380
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
- 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
- 2017Temporal Action Detection with Structured Segment Networkscited by 265
- 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
- 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
- 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 141
- 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
- 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
- 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112
- 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
- 2020Surgical Data Science -- from Concepts toward Clinical Translationcited by 105
- 2018Diagnosing Error in Temporal Action Detectorscited by 71
- 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
- 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
- 2018DeepPhase: Surgical Phase Recognition in CATARACTS Videoscited by 47
- 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 31
- 2024DyFADet: Dynamic Feature Aggregation for Temporal Action Detectioncited by 18
- 2024MuST: Multi-Scale Transformers for Surgical Phase Recognitioncited by 8
- 2024Temporal Action Localization with Cross Layer Task Decoupling and Refinementcited by 3
Built on by
Not cited yet.