# An autoencoder compression approach for accelerating large-scale inverse problems

arXiv: https://arxiv.org/abs/2304.04781 · HTML: https://mlsanity.com/paper/2304.04781/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 78 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [Deep Convolutional AutoEncoder-based Lossy Image Compression](https://mlsanity.com/paper/1804.09535/) (2018)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [Solving Bayesian Inverse Problems via Variational Autoencoders](https://mlsanity.com/paper/1912.04212/) (2019)
- [Better Latent Spaces for Better Autoencoders](https://mlsanity.com/paper/2104.08291/) (2021)
- [An autoencoder-based reduced-order model for eigenvalue problems with application to neutron diffusion](https://mlsanity.com/paper/2008.10532/) (2020)
- [Exploring Autoencoder-based Error-bounded Compression for Scientific Data](https://mlsanity.com/paper/2105.11730/) (2021)
- [SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation](https://mlsanity.com/paper/2005.03403/) (2020)
- [Symmetry-Aware Autoencoders: s-PCA and s-nlPCA](https://mlsanity.com/paper/2111.02893/) (2021)
- [A PCA-like Autoencoder](https://mlsanity.com/paper/1904.01277/) (2019)
- [Closing the "Quantum Supremacy" Gap: Achieving Real-Time Simulation of a Random Quantum Circuit Using a New Sunway Supercomputer](https://mlsanity.com/paper/2110.14502/) (2021)
- [Early Evaluation of Intel Optane Non-Volatile Memory with HPC I/O Workloads](https://mlsanity.com/paper/1708.02199/) (2017)
- [Efficient Data Compression with Error Bound Guarantee in Wireless Sensor Networks](https://mlsanity.com/paper/1408.2948/) (2014)
- [Trading Computation for Communication: A Taxonomy](https://mlsanity.com/paper/1709.06555/) (2017)
- [Combining Checkpointing and Data Compression to Accelerate Adjoint-Based Optimization Problems](https://mlsanity.com/paper/1810.05268/) (2018)
- [Demystifying the Performance of HPC Scientific Applications on NVM-based Memory Systems](https://mlsanity.com/paper/2002.06499/) (2020)

## Built on by (most cited first, 1 of 1)

- [Likelihood-Scheduled Score-Based Generative Modeling for Fully 3D PET Image Reconstruction](https://mlsanity.com/paper/2412.04339/) (2024)

## Abstract (arXiv)

PDE-constrained inverse problems are some of the most challenging and computationally demanding problems in computational science today. Fine meshes that are required to accurately compute the PDE solution introduce an enormous number of parameters and require large scale computing resources such as more processors and more memory to solve such systems in a reasonable time. For inverse problems constrained by time dependent PDEs, the adjoint method that is often employed to efficiently compute gradients and higher order derivatives requires solving a time-reversed, so-called adjoint PDE that depends on the forward PDE solution at each timestep. This necessitates the storage of a high dimensional forward solution vector at every timestep. Such a procedure quickly exhausts the available memory resources. Several approaches that trade additional computation for reduced memory footprint have been proposed to mitigate the memory bottleneck, including checkpointing and compression strategies. In this work, we propose a close-to-ideal scalable compression approach using autoencoders to eliminate the need for checkpointing and substantial memory storage, thereby reducing both the time-to-solution and memory requirements. We compare our approach with checkpointing and an off-the-shelf compression approach on an earth-scale ill-posed seismic inverse problem. The results verify the expected close-to-ideal speedup for both the gradient and Hessian-vector product using the proposed autoencoder compression approach. To highlight the usefulness of the proposed approach, we combine the autoencoder compression with the data-informed active subspace (DIAS) prior to show how the DIAS method can be affordably extended to large scale problems without the need of checkpointing and large memory.