# SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

arXiv: https://arxiv.org/abs/2608.08070 · HTML: https://mlsanity.com/paper/2608.08070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical and surgical world models, one of 11 directions in World models.
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Placed in

- World models / [Medical and surgical world models](https://mlsanity.com/wm/medical-and-surgical-world-models/)

## Builds on (most cited first, 8 of 8)

- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge](https://mlsanity.com/paper/2401.00496/) (2023)
- [Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks](https://mlsanity.com/paper/2407.12998/) (2024)
- [Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets](https://mlsanity.com/paper/2204.05235/) (2022)
- [Cosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modeling](https://mlsanity.com/paper/2512.23162/) (2025)
- [SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation](https://mlsanity.com/paper/2603.13024/) (2026)
- [SutureFormer: Learning Surgical Trajectories via Goal-conditioned Offline RL in Pixel Space](https://mlsanity.com/paper/2603.26720/) (2026)

## Built on by (most cited first, 1 of 1)

- [Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning](https://mlsanity.com/paper/2608.20284/) (2026)

## Abstract (arXiv)

Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead anticipate how instrument motion reshapes subsequent operative states. Most surgical video understanding methods focus on recognizing phases, actions, or workflow states, while providing limited support for explicitly modeling instrument motion. Conversely, existing tool motion prediction methods can forecast instrument trajectories, but they generally do not capture the coupled evolution of future surgical video states. World models offer a natural framework for jointly modeling visual state transitions and instrument motion dynamics. However, existing surgical world model studies remain largely centered on visual generation quality, relying on generation-oriented metrics such as FVD and CD-FVD. These metrics are poorly aligned with instrument motion planning, as they do not directly measure whether predicted trajectories are geometrically accurate, temporally coherent, or actionable for downstream planning. This limitation is partly structural, since the field lacks public datasets and standardized evaluation protocols that provide the benchmarking infrastructure needed to assess motion-centric capabilities in surgical world models. In this paper, we introduce SurgWMBench, a vision-based benchmark for short-horizon surgical motion planning and dynamics prediction. Given intraoperative image sequences and historical instrument trajectory, SurgWMBench evaluates both near-future instrument motion prediction and stability under continuous rollout or input perturbations.