mlsanityData to 7 Oct 2026

OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality

arXiv 2609.12409 ↗First posted 11 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

High-frequency surgical maneuvers in ophthalmology necessitate high-fidelity temporal modeling, yet characterizing long-range procedural dependencies remains computationally prohibitive for attention-based architectures. Existing models often require aggressive temporal downsampling, which compromises the detection of fine-grained action boundaries and instrument-tissue interactions. To address these scalability constraints, we present OphBiWSSD, a framework that reformulates surgical temporal action localization leveraging Bidirectional State Space Duality. By employing a weight-tied selective scan mechanism that incorporates both preceding and succeeding surgical contexts, our approach facilitates the global synthesis of non-causal temporal cues with linear complexity. This streamlined architecture is well-suited to capture the bidirectional dependencies present in ophthalmic workflows, effectively bridging the gap between local boundary precision and long-range procedural context without incurring the quadratic memory overhead of traditional Transformers. Extensive experiments on the OphNet benchmark demonstrate that OphBiWSSD achieves state-of-the-art temporal localization performance, with mean Average Precisions of 44.42% on phases and 43.08% on operations, surpassing the baselines by 6.80% and 6.66%, respectively. Empirical validation indicates that our approach ensures precise temporal localization and offers a computationally viable pathway for deploying surgical intelligence systems in clinical environments. The code is publicly available at https://github.com/yo3nglau/OphBiWSSD.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

34 of 38 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2018SlowFast Networks for Video Recognitioncited by 1,862
  5. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  6. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  7. 2021Video Swin Transformercited by 1,090
  8. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  9. 2016Deep Networks with Stochastic Depthcited by 973
  10. 2024VMamba: Visual State Space Modelcited by 955
  11. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  12. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
22 more
  1. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  2. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 380
  3. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  4. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  5. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  6. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  7. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  8. 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
  9. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 141
  10. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  11. 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
  12. 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112
  13. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  14. 2020Surgical Data Science -- from Concepts toward Clinical Translationcited by 105
  15. 2018Diagnosing Error in Temporal Action Detectorscited by 71
  16. 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
  17. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
  18. 2018DeepPhase: Surgical Phase Recognition in CATARACTS Videoscited by 47
  19. 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 31
  20. 2024DyFADet: Dynamic Feature Aggregation for Temporal Action Detectioncited by 18
  21. 2024MuST: Multi-Scale Transformers for Surgical Phase Recognitioncited by 8
  22. 2024Temporal Action Localization with Cross Layer Task Decoupling and Refinementcited by 3

Built on by

Not cited yet.