# DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets

arXiv: https://arxiv.org/abs/2303.03403 · HTML: https://mlsanity.com/paper/2303.03403/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Reconstruction of three-dimensional porous media using generative adversarial neural networks](https://mlsanity.com/paper/1704.03225/) (2017)
- [Generating 3D structures from a 2D slice with GAN-based dimensionality expansion](https://mlsanity.com/paper/2102.07708/) (2021)
- [A Transfer Learning Approach for Microstructure Reconstruction and Structure-property Predictions](https://mlsanity.com/paper/1805.02784/) (2018)
- [Microstructure Representation and Reconstruction of Heterogeneous Materials via Deep Belief Network for Computational Material Design](https://mlsanity.com/paper/1612.07401/) (2016)
- [A Tutorial on VAEs: From Bayes' Rule to Lossless Compression](https://mlsanity.com/paper/2006.10273/) (2020)
- [Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics](https://mlsanity.com/paper/2206.01693/) (2022)
- [An efficient optimization based microstructure reconstruction approach with multiple loss functions](https://mlsanity.com/paper/2102.02407/) (2021)
- [RockGPT: Reconstructing three-dimensional digital rocks from single two-dimensional slice from the perspective of video generation](https://mlsanity.com/paper/2108.03132/) (2021)

## Built on by (most cited first, 5 of 5)

- [Deep Learning in Deterministic Computational Mechanics](https://mlsanity.com/paper/2309.15421/) (2023)
- [Deep learning-aided inverse design of porous metamaterials](https://mlsanity.com/paper/2507.17907/) (2025)
- [Toward Learning Latent-Variable Representations of Microstructures by Optimizing in Spatial Statistics Space](https://mlsanity.com/paper/2402.11103/) (2024)
- [A Decade of Deep Learning: A Survey on The Magnificent Seven](https://mlsanity.com/paper/2412.16188/) (2024)
- [Physics-Guided Generative AI for Property-Targeted 3D Porous Media Design](https://mlsanity.com/paper/2607.24274/) (2026)

## Abstract (arXiv)

Microstructure reconstruction is an important and emerging field of research and an essential foundation to improving inverse computational materials engineering (ICME). Much of the recent progress in the field is made based on generative adversarial networks (GANs). Although excellent results have been achieved throughout a variety of materials, challenges remain regarding the interpretability of the model's latent space as well as the applicability to extremely small data sets. The present work addresses these issues by introducing DA-VEGAN, a model with two central innovations. First, a β-variational autoencoder is incorporated into a hybrid GAN architecture that allows to penalize strong nonlinearities in the latent space by an additional parameter, β. Secondly, a custom differentiable data augmentation scheme is developed specifically for this architecture. The differentiability allows the model to learn from extremely small data sets without mode collapse or deteriorated sample quality. An extensive validation on a variety of structures demonstrates the potential of the method and future directions of investigation are discussed.