# Learning Soft Tissue Behavior of Organs for Surgical Navigation with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1904.00722 · HTML: https://mlsanity.com/paper/1904.00722/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 19 of 19)

- [PhysGNN: A Physics-Driven Graph Neural Network Based Model for Predicting Soft Tissue Deformation in Image-Guided Neurosurgery](https://mlsanity.com/paper/2109.04352/) (2021)
- [A Physics-Guided Neural Operator Learning Approach to Model Biological Tissues from Digital Image Correlation Measurements](https://mlsanity.com/paper/2204.00205/) (2022)
- [DeepPhysics: a physics aware deep learning framework for real-time simulation](https://mlsanity.com/paper/2109.09491/) (2021)
- [Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning](https://mlsanity.com/paper/2301.04614/) (2023)
- [Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics](https://mlsanity.com/paper/2212.01386/) (2022)
- [Anatomy-informed Data Augmentation for Enhanced Prostate Cancer Detection](https://mlsanity.com/paper/2309.03652/) (2023)
- [A Review on Organ Deformation Modeling Approaches for Reliable Surgical Navigation using Augmented Reality](https://mlsanity.com/paper/2408.02713/) (2024)
- [Heterogeneous Peridynamic Neural Operators: Discover Biotissue Constitutive Law and Microstructure From Digital Image Correlation Measurements](https://mlsanity.com/paper/2403.18597/) (2024)
- [LUDO: Low-Latency Understanding of Deformable Objects using Point Cloud Occupancy Functions](https://mlsanity.com/paper/2411.08777/) (2024)
- [Toward Reliable AR-Guided Surgical Navigation: Interactive Deformation Modeling with Data-Driven Biomechanics and Prompts](https://mlsanity.com/paper/2506.08048/) (2025)
- [Analytically Integratable Zero-restlength Springs for Capturing Dynamic Modes unrepresented by Quasistatic Neural Networks](https://mlsanity.com/paper/2201.10122/) (2022)
- [Kernel-based framework to estimate deformations of pneumothorax lung using relative position of anatomical landmarks](https://mlsanity.com/paper/2102.12505/) (2021)
- [Neural-Augmented Kelvinlet for Real-Time Soft Tissue Deformation Modeling](https://mlsanity.com/paper/2506.08043/) (2025)
- [Conditional Graph Neural Network for Predicting Soft Tissue Deformation and Forces](https://mlsanity.com/paper/2507.05315/) (2025)
- [PIVOTS: Aligning unseen Structures using Preoperative to Intraoperative Volume-To-Surface Registration for Liver Navigation](https://mlsanity.com/paper/2507.20337/) (2025)
- [From Pre- to Intra-operative MRI: Predicting Brain Shift in Temporal Lobe Resection for Epilepsy Surgery](https://mlsanity.com/paper/2602.03785/) (2026)
- [Data-driven registration and modeling of brain deformation for image-guided neurosurgery](https://mlsanity.com/paper/2602.10155/) (2026)
- [Deep learning based Non-Rigid Volume-to-Surface Registration for Brain Shift compensation Using Point Cloud](https://mlsanity.com/paper/2604.17389/) (2026)
- [DeepFEAv2: Deep Learning for Transient Finite Element Analysis Beyond Structured Meshes](https://mlsanity.com/paper/2609.26426/) (2026)

## Abstract (arXiv)

Purpose: In surgical navigation, pre-operative organ models are presented to surgeons during the intervention to help them in efficiently finding their target. In the case of soft tissue, these models need to be deformed and adapted to the current situation by using intra-operative sensor data. A promising method to realize this are real-time capable biomechanical models. Methods: We train a fully convolutional neural network to estimate a displacement field of all points inside an organ when given only the displacement of a part of the organ's surface. The network trains on entirely synthetic data of random organ-like meshes, which allows us to generate much more data than is otherwise available. The input and output data is discretized into a regular grid, allowing us to fully utilize the capabilities of convolutional operators and to train and infer in a highly parallelized manner. Results: The system is evaluated on in-silico liver models, phantom liver data and human in-vivo breathing data. We test the performance with varying material parameters, organ shapes and amount of visible surface. Even though the network is only trained on synthetic data, it adapts well to the various cases and gives a good estimation of the internal organ displacement. The inference runs at over 50 frames per second. Conclusions: We present a novel method for training a data-driven, real-time capable deformation model. The accuracy is comparable to other registration methods, it adapts very well to previously unseen organs and does not need to be re-trained for every patient. The high inferring speed makes this method useful for many applications such as surgical navigation and real-time simulation.