# Real-time Nonrigid Mosaicking of Laparoscopy Images

arXiv: https://arxiv.org/abs/2103.07414 · HTML: https://mlsanity.com/paper/2103.07414/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 49 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [VolumeDeform: Real-time Volumetric Non-rigid Reconstruction](https://mlsanity.com/paper/1603.08161/) (2016)
- [DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences](https://mlsanity.com/paper/1908.08918/) (2019)
- [MIS-SLAM: Real-time Large Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing](https://mlsanity.com/paper/1803.02009/) (2018)
- [Guided Stereo Matching](https://mlsanity.com/paper/1905.10107/) (2019)
- [Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video](https://mlsanity.com/paper/2007.12623/) (2020)
- [Deep Placental Vessel Segmentation for Fetoscopic Mosaicking](https://mlsanity.com/paper/2007.04349/) (2020)
- [Quasi-homography warps in image stitching](https://mlsanity.com/paper/1701.08006/) (2017)
- [Image computing for fibre-bundle endomicroscopy: A review](https://mlsanity.com/paper/1809.00604/) (2018)
- [Understanding Soft-Tissue Behavior for Application to Microlaparoscopic Surface Scan](https://mlsanity.com/paper/1804.04969/) (2018)
- [Smooth Deformation Field-based Mismatch Removal in Real-time](https://mlsanity.com/paper/2007.08553/) (2020)

## Built on by (most cited first, 5 of 5)

- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [BDIS-SLAM: A lightweight CPU-based dense stereo SLAM for surgery](https://mlsanity.com/paper/2312.15679/) (2023)
- [Learning-Based Keypoint Registration for Fetoscopic Mosaicking](https://mlsanity.com/paper/2207.13185/) (2022)
- [Towards Abdominal 3-D Scene Rendering from Laparoscopy Surgical Videos using NeRFs](https://mlsanity.com/paper/2310.11645/) (2023)
- [FloVMos: Optical Flow-based Medical Video Mosaicking](https://mlsanity.com/paper/2610.04258/) (2026)

## Abstract (arXiv)

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on textitin vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.