# Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data

arXiv: https://arxiv.org/abs/2407.00595 · HTML: https://mlsanity.com/paper/2407.00595/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 30 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [The Reversible Residual Network: Backpropagation Without Storing Activations](https://mlsanity.com/paper/1707.04585/) (2017)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [i-RevNet: Deep Invertible Networks](https://mlsanity.com/paper/1802.07088/) (2018)
- [Reversible Architectures for Arbitrarily Deep Residual Neural Networks](https://mlsanity.com/paper/1709.03698/) (2017)
- [Constrained-CNN losses for weakly supervised segmentation](https://mlsanity.com/paper/1805.04628/) (2018)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [Fully Hyperbolic Convolutional Neural Networks](https://mlsanity.com/paper/1905.10484/) (2019)
- [A Regularized Convolutional Neural Network for Semantic Image Segmentation](https://mlsanity.com/paper/1907.05287/) (2019)
- [Compression of Fully-Connected Layer in Neural Network by Kronecker Product](https://mlsanity.com/paper/1507.05775/) (2015)
- [MD Loss: Efficient Training of 3D Seismic Fault Segmentation Network under Sparse Labels by Weakening Anomaly Annotation](https://mlsanity.com/paper/2110.05319/) (2021)
- [InvertibleNetworks.jl: A Julia package for scalable normalizing flows](https://mlsanity.com/paper/2312.13480/) (2023)
- [Multi-resolution neural networks for tracking seismic horizons from few training images](https://mlsanity.com/paper/1812.11092/) (2018)
- [Fully reversible neural networks for large-scale 3D seismic horizon tracking](https://mlsanity.com/paper/2003.08466/) (2020)
- [LeanResNet: A Low-cost Yet Effective Convolutional Residual Networks](https://mlsanity.com/paper/1904.06952/) (2019)
- [Symmetric block-low-rank layers for fully reversible multilevel neural networks](https://mlsanity.com/paper/1912.12137/) (2019)
- [Fast methods for denoising matrix completion formulations, with applications to robust seismic data interpolation](https://mlsanity.com/paper/1302.4886/) (2013)
- [Neural-networks for geophysicists and their application to seismic data interpretation](https://mlsanity.com/paper/1903.11215/) (2019)

## Abstract (arXiv)

The large spatial/temporal/frequency scale of geoscience and remote-sensing datasets causes memory issues when using convolutional neural networks for (sub-) surface data segmentation. Recently developed fully reversible or fully invertible networks can mostly avoid memory limitations by recomputing the states during the backward pass through the network. This results in a low and fixed memory requirement for storing network states, as opposed to the typical linear memory growth with network depth. This work focuses on a fully invertible network based on the telegraph equation. While reversibility saves the major amount of memory used in deep networks by the data, the convolutional kernels can take up most memory if fully invertible networks contain multiple invertible pooling/coarsening layers. We address the explosion of the number of convolutional kernels by combining fully invertible networks with layers that contain the convolutional kernels in a compressed form directly. A second challenge is that invertible networks output a tensor the same size as its input. This property prevents the straightforward application of invertible networks to applications that map between different input-output dimensions, need to map to outputs with more channels than present in the input data, or desire outputs that decrease/increase the resolution compared to the input data. However, we show that by employing invertible networks in a non-standard fashion, we can still use them for these tasks. Examples in hyperspectral land-use classification, airborne geophysical surveying, and seismic imaging illustrate that we can input large data volumes in one chunk and do not need to work on small patches, use dimensionality reduction, or employ methods that classify a patch to a single central pixel.