# EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-training arXiv: https://arxiv.org/abs/2502.19090 ยท HTML: https://mlsanity.com/paper/2502.19090/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Mamba for medical image analysis, one of 10 directions in State-space sequence models (S4, Mamba). - Builds on 27 of its 37 references that are in the map. - First posted on arXiv 26 Feb 2025. ## Placed in - State-space sequence models (S4, Mamba) / [Mamba for medical image analysis](https://mlsanity.com/ssm/mamba-for-medical-image-analysis/) ## Builds on (most cited first, 27 of 27) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024) - [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022) - [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024) - [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023) - [VideoMamba: State Space Model for Efficient Video Understanding](https://mlsanity.com/paper/2403.06977/) (2024) - [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024) - [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023) - [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021) - [Unmasked Teacher: Towards Training-Efficient Video Foundation Models](https://mlsanity.com/paper/2303.16058/) (2023) - [ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning](https://mlsanity.com/paper/2206.13559/) (2022) - [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021) - [Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer](https://mlsanity.com/paper/2103.09712/) (2021) - [AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomy](https://mlsanity.com/paper/2208.02049/) (2022) - [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022) - [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023) - [LoViT: Long Video Transformer for Surgical Phase Recognition](https://mlsanity.com/paper/2305.08989/) (2023) - [EndoMapper dataset of complete calibrated endoscopy procedures](https://mlsanity.com/paper/2204.14240/) (2022) - [Motion-aware Contrastive Video Representation Learning via Foreground-background Merging](https://mlsanity.com/paper/2109.15130/) (2021) - [Probabilistic Representations for Video Contrastive Learning](https://mlsanity.com/paper/2204.03946/) (2022) - [Static and Dynamic Concepts for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2207.12795/) (2022) - [Contrastive Transformer-based Multiple Instance Learning for Weakly Supervised Polyp Frame Detection](https://mlsanity.com/paper/2203.12121/) (2022) - [Autonomous Driving in the Lung using Deep Learning for Localization](https://mlsanity.com/paper/1907.08136/) (2019) - [EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels](https://mlsanity.com/paper/2409.05442/) (2024) - [BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization](https://mlsanity.com/paper/2402.12763/) (2024) - [hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries](https://mlsanity.com/paper/2110.12555/) (2021) ## Built on by (most cited first, 11 of 11) - [FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video Generation](https://mlsanity.com/paper/2506.04956/) (2025) - [SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos](https://mlsanity.com/paper/2602.05638/) (2026) - [EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis](https://mlsanity.com/paper/2508.07292/) (2025) - [BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion](https://mlsanity.com/paper/2601.03713/) (2026) - [Harnessing Foundation Models for Robust and Generalizable 6-DOF Bronchoscopy Localization](https://mlsanity.com/paper/2505.24249/) (2025) - [Colon-Bench: An Agentic Workflow for Scalable Dense Lesion Annotation in Full-Procedure Colonoscopy Videos](https://mlsanity.com/paper/2603.25645/) (2026) - [EndoMatcher: Generalizable Endoscopic Image Matcher via Multi-Domain Pre-training for Robot-Assisted Surgery](https://mlsanity.com/paper/2508.05205/) (2025) - [SurgicalMamba: Dual-Path SSD with State Regramming for Online Surgical Phase Recognition](https://mlsanity.com/paper/2605.14889/) (2026) - [Geometry-Consistent Endoscopic Representations for Image-Guided Navigation via Structured Foundation Model Adaptation](https://mlsanity.com/paper/2606.17340/) (2026) - [EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment](https://mlsanity.com/paper/2608.04472/) (2026) - [Representation-driven Endoscopic Visual Embedding Alignment for Latent Generation](https://mlsanity.com/paper/2608.07176/) (2026) ## Abstract (arXiv) Endoscopic video-based tasks, such as visual navigation and surgical phase recognition, play a crucial role in minimally invasive surgeries by providing real-time assistance. While recent video foundation models have shown promise, their applications are hindered by (1) computational inefficiencies and (2) suboptimal performance caused by limited data for pre-training in endoscopy. To address these issues, we present EndoMamba, a foundation model designed for real-time inference while learning generalized spatiotemporal representations. First, to mitigate computational inefficiencies, we propose the EndoMamba backbone, optimized for real-time inference. Inspired by recent advancements in state space models, EndoMamba integrates Bidirectional Mamba blocks for spatial modeling within individual frames and vanilla Mamba blocks for past-to-present reasoning across the temporal domain. This design enables both strong spatiotemporal modeling and efficient inference in online video streams. Second, we propose a self-supervised hierarchical pre-training diagram to enhance EndoMamba's representation learning using endoscopic videos and incorporating general video domain knowledge. Specifically, our approach combines masked reconstruction with auxiliary supervision, leveraging low-level reconstruction to capture spatial-temporal structures and high-level alignment to transfer broader knowledge from a pretrained general-video domain foundation model. Extensive experiments on four downstream tasks--classification, segmentation, surgical phase recognition, and localization--demonstrate that EndoMamba outperforms existing foundation models and task-specific methods while maintaining real-time inference speed. The source code is available at https://github.com/TianCuteQY/EndoMamba.