# EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control

arXiv: https://arxiv.org/abs/2507.15292 · HTML: https://mlsanity.com/paper/2507.15292/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Motion Prompting: Controlling Video Generation with Motion Trajectories](https://mlsanity.com/paper/2412.02700/) (2024)
- [MFT: Long-Term Tracking of Every Pixel](https://mlsanity.com/paper/2305.12998/) (2023)
- [Pixel-Wise Recognition for Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2401.11174/) (2024)
- [Learning-based Video Motion Magnification](https://mlsanity.com/paper/1804.02684/) (2018)
- [EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer](https://mlsanity.com/paper/2312.04152/) (2023)
- [Motion-Conditioned Image Animation for Video Editing](https://mlsanity.com/paper/2311.18827/) (2023)
- [Robust Medical Instrument Segmentation Challenge 2019](https://mlsanity.com/paper/2003.10299/) (2020)
- [Video Acceleration Magnification](https://mlsanity.com/paper/1704.04186/) (2017)
- [UniVST: A Unified Framework for Training-free Localized Video Style Transfer](https://mlsanity.com/paper/2410.20084/) (2024)
- [S2ME: Spatial-Spectral Mutual Teaching and Ensemble Learning for Scribble-supervised Polyp Segmentation](https://mlsanity.com/paper/2306.00451/) (2023)
- [EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation](https://mlsanity.com/paper/2408.13005/) (2024)
- [Self-Supervised Motion Magnification by Backpropagating Through Optical Flow](https://mlsanity.com/paper/2311.17056/) (2023)
- [Deep Video Harmonization with Color Mapping Consistency](https://mlsanity.com/paper/2205.00687/) (2022)
- [Motion Magnification in Robotic Sonography: Enabling Pulsation-Aware Artery Segmentation](https://mlsanity.com/paper/2307.03698/) (2023)
- [SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge](https://mlsanity.com/paper/2407.11906/) (2024)
- [Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition](https://mlsanity.com/paper/2204.07636/) (2022)
- [Weakly Supervised Temporal Convolutional Networks for Fine-grained Surgical Activity Recognition](https://mlsanity.com/paper/2302.10834/) (2023)
- [STB-VMM: Swin Transformer Based Video Motion Magnification](https://mlsanity.com/paper/2302.10001/) (2023)
- [Learning-based Axial Video Motion Magnification](https://mlsanity.com/paper/2312.09551/) (2023)
- [Surgical Video Motion Magnification with Suppression of Instrument Artefacts](https://mlsanity.com/paper/2009.07432/) (2020)
- [MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulation](https://mlsanity.com/paper/2412.19978/) (2024)
- [Higher Order of Motion Magnification for Vessel Localisation in Surgical Video](https://mlsanity.com/paper/1806.04955/) (2018)

## Abstract (arXiv)

Visualizing subtle vascular motions in endoscopic surgery is crucial for surgical precision and decision-making, yet remains challenging due to the complex and dynamic nature of surgical scenes. To address this, we introduce EndoControlMag, a training-free, Lagrangian-based framework with mask-conditioned vascular motion magnification tailored to endoscopic environments. Our approach features two key modules: a Periodic Reference Resetting (PRR) scheme that divides videos into short overlapping clips with dynamically updated reference frames to prevent error accumulation while maintaining temporal coherence, and a Hierarchical Tissue-aware Magnification (HTM) framework with dual-mode mask dilation. HTM first tracks vessel cores using a pretrained visual tracking model to maintain accurate localization despite occlusions and view changes. It then applies one of two adaptive softening strategies to surrounding tissues: motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This dual-mode approach accommodates diverse surgical scenarios-motion-based softening excels with complex tissue deformations while distance-based softening provides stability during unreliable optical flow conditions. We evaluate EndoControlMag on our EndoVMM24 dataset spanning four different surgery types and various challenging scenarios, including occlusions, instrument disturbance, view changes, and vessel deformations. Quantitative metrics, visual assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in both magnification accuracy and visual quality while maintaining robustness across challenging surgical conditions. The code, dataset, and video results are available at https://szupc.github.io/EndoControlMag/.