# Generative Optimization Networks for Memory Efficient Data Generation

arXiv: https://arxiv.org/abs/2110.02912 · HTML: https://mlsanity.com/paper/2110.02912/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks](https://mlsanity.com/paper/1901.04997/) (2019)
- [Communication-Computation Trade-Off in Resource-Constrained Edge Inference](https://mlsanity.com/paper/2006.02166/) (2020)
- [GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework](https://mlsanity.com/paper/2008.11062/) (2020)
- [Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals](https://mlsanity.com/paper/2107.12626/) (2021)
- [Slimmable Generative Adversarial Networks](https://mlsanity.com/paper/2012.05660/) (2020)
- [COSCO: Container Orchestration using Co-Simulation and Gradient Based Optimization for Fog Computing Environments](https://mlsanity.com/paper/2104.14392/) (2021)
- [A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices](https://mlsanity.com/paper/1907.10147/) (2019)
- [Self-supervised Adversarial Training](https://mlsanity.com/paper/1911.06470/) (2019)
- [DeFog: Fog Computing Benchmarks](https://mlsanity.com/paper/1907.10890/) (2019)
- [Real-Time Sensor Anomaly Detection and Recovery in Connected Automated Vehicle Sensors](https://mlsanity.com/paper/1911.01531/) (2019)

## Built on by (most cited first, 5 of 5)

- [GOSH: Task Scheduling Using Deep Surrogate Models in Fog Computing Environments](https://mlsanity.com/paper/2112.08916/) (2021)
- [MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems](https://mlsanity.com/paper/2112.07269/) (2021)
- [Can We Find Strong Lottery Tickets in Generative Models?](https://mlsanity.com/paper/2212.08311/) (2022)
- [CAROL: Confidence-Aware Resilience Model for Edge Federations](https://mlsanity.com/paper/2203.07140/) (2022)
- [DRAGON: Decentralized Fault Tolerance in Edge Federations](https://mlsanity.com/paper/2208.07658/) (2022)

## Abstract (arXiv)

In standard generative deep learning models, such as autoencoders or GANs, the size of the parameter set is proportional to the complexity of the generated data distribution. A significant challenge is to deploy resource-hungry deep learning models in devices with limited memory to prevent system upgrade costs. To combat this, we propose a novel framework called generative optimization networks (GON) that is similar to GANs, but does not use a generator, significantly reducing its memory footprint. GONs use a single discriminator network and run optimization in the input space to generate new data samples, achieving an effective compromise between training time and memory consumption. GONs are most suited for data generation problems in limited memory settings. Here we illustrate their use for the problem of anomaly detection in memory-constrained edge devices arising from attacks or intrusion events. Specifically, we use a GON to calculate a reconstruction-based anomaly score for input time-series windows. Experiments on a Raspberry-Pi testbed with two existing and a new suite of datasets show that our framework gives up to 32% higher detection F1 scores and 58% lower memory consumption, with only 5% higher training overheads compared to the state-of-the-art.