SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos
Not on the map. Why?
Abstract (arXiv)
While foundation models have advanced surgical video analysis, current approaches rely predominantly on pixel-level reconstruction objectives that waste model capacity on low-level visual details, such as smoke, specular reflections, and fluid motion, rather than semantic structures essential for surgical understanding. We present SurgMotion, a video-native foundation model that shifts the learning paradigm from pixel-level reconstruction to latent motion prediction. Built on the Video Joint Embedding Predictive Architecture (V-JEPA), SurgMotion introduces three key technical innovations tailored to surgical videos: (1) motion-guided latent masked prediction to prioritize semantically meaningful regions, (2) spatiotemporal affinity self-distillation to enforce relational consistency, and (3) spatiotemporal feature diversity regularization (SFDR) to prevent representation collapse in texture-sparse surgical scenes. To enable large-scale pretraining, we curate SurgMotion-15M, the largest surgical video dataset to date, comprising 3,658 hours of video from 50 sources across 13 anatomical regions. Extensive experiments across 17 benchmarks demonstrate that SurgMotion significantly outperforms state-of-the-art methods on surgical workflow recognition, achieving 14.6 percent improvement in F1 score on EgoSurgery and 10.3 percent on PitVis; on action triplet recognition with 39.54 percent mAP-IVT on CholecT50; as well as on skill assessment, polyp segmentation, and depth estimation. These results establish SurgMotion as a new standard for universal, motion-oriented surgical video understanding.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 5 Feb 2026.
Builds on
36 of 48 references.
References · 36
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2025DINOv3cited by 1,414
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 589
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
- 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
- 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
24 more
- 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 367
- 2016A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Imagescited by 221
- 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
- 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
- 2022Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registrationcited by 80
- 2023Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-traincited by 75
- 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
- 2021UACANet: Uncertainty Augmented Context Attention for Polyp Segmentationcited by 61
- 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
- 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
- 2016The TUM LapChole dataset for the M2CAI 2016 workflow challengecited by 38
- 2022Towards Holistic Surgical Scene Understandingcited by 33
- 2024OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understandingcited by 32
- 2024General surgery vision transformer: A video pre-trained foundation model for general surgerycited by 25
- 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
- 2023Self-Supervised Learning for Endoscopic Video Analysiscited by 22
- 2022Contrastive Transformer-based Multiple Instance Learning for Weakly Supervised Polyp Frame Detectioncited by 20
- 2024EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videoscited by 20
- 2024CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissectioncited by 15
- 2024Surgical Workflow Recognition and Blocking Effectiveness Detection in Laparoscopic Liver Resections with Pringle Maneuvercited by 12
- 2025EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-trainingcited by 11
- 2024PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgerycited by 10
- 2025InternVideo-Next: Towards General Video Foundation Models without Video-Text Supervisioncited by 7
- 2025PraNet-V2: Dual-Supervised Reverse Attention for Medical Image Segmentationcited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligencecited by 2
- 2026CrossScope: A Role-Asymmetric World Model for Joint Dual-Scope Surgical Video Predictioncited by 0
- 2026Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessmentcited by 0
- 2026Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planningcited by 0
- 2026Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning20 Aug 2026
- 2026Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment18 Aug 2026
- 2026CrossScope: A Role-Asymmetric World Model for Joint Dual-Scope Surgical Video Prediction4 Aug 2026
- 2026SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence17 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Scaling Video Pretraining for Surgical Foundation Models
- 2025Large-scale Self-supervised Video Foundation Model for Intelligent Surgery
- 2025SurgLLM: A Versatile Large Multimodal Model with Spatial Focus and Temporal Awareness for Surgical Video Understanding
- 2026An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models
- 2026SurgNarrator: A Generative Retrieval Framework for Surgical Video Understanding
- 2025Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model
- 2026SurgGraph: Quantitative Laparoscopic Video Understanding via Geometry-Grounded Scene Graphs ↗
- 2025SurgVidLM: Towards Multi-grained Surgical Video Understanding with Large Language Model