mlsanityData to 7 Oct 2026

Feature Alignment as a Generative Process

arXiv 2106.12562 ↗First posted 23 Jun 2021cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reversibility in artificial neural networks allows us to retrieve the input given an output. We present feature alignment, a method for approximating reversibility in arbitrary neural networks. We train a network by minimizing the distance between the output of a data point and the random output with respect to a random input. We applied the technique to the MNIST, CIFAR-10, CelebA and STL-10 image datasets. We demonstrate that this method can roughly recover images from just their latent representation without the need of a decoder. By utilizing the formulation of variational autoencoders, we demonstrate that it is possible to produce new images that are statistically comparable to the training data. Furthermore, we demonstrate that the quality of the images can be improved by coupling a generator and a discriminator together. In addition, we show how this method, with a few minor modifications, can be used to train networks locally, which has the potential to save computational memory resources.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 57 of its 93 references that are in the map. First posted on arXiv 23 Jun 2021.

Builds on

57 of 93 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  11. 2016Least Squares Generative Adversarial Networkscited by 1,000
  12. 2019An Introduction to Variational Autoencoderscited by 997
38 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  2. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  3. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  4. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  5. 2017Wasserstein GANcited by 635
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  7. 2017Large Batch Training of Convolutional Networkscited by 529
  8. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2016Tutorial on Variational Autoencoderscited by 404
  11. 2018Invertible Residual Networkscited by 364
  12. 2016Adversarial Feature Learningcited by 350
  13. 2018Understanding disentangling in β-VAEcited by 332
  14. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 239
  15. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 221
  16. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 215
  17. 2020A Review on Generative Adversarial Networks: Algorithms, Theory, and Applicationscited by 209
  18. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  19. 2019Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapsecited by 120
  20. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  21. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  22. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88
  23. 2018Taming VAEscited by 85
  24. 2014How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagationcited by 77
  25. 2020Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networkscited by 73
  26. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  27. 2019Low-Memory Neural Network Training: A Technical Reportcited by 56
  28. 2018Unsupervised Learning by Competing Hidden Unitscited by 44
  29. 2019Neural Density Estimation and Likelihood-free Inferencecited by 23
  30. 2020Parallel Training of Deep Networks with Local Updatescited by 18
  31. 2021A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networkscited by 14
  32. 2021Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networkscited by 14
  33. 2018Conducting Credit Assignment by Aligning Local Representationscited by 12
  34. 2019A Survey on Neural Network Language Modelscited by 12
  35. 2020Preventing Posterior Collapse with Levenshtein Variational Autoencodercited by 12
  36. 2015A Theory of Local Learning, the Learning Channel, and the Optimality of Backpropagationcited by 11
  37. 2020Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developmentscited by 11
  38. 2020Neural Network for Low-Memory IoT Devices and MNIST Image Recognition Using Kernels Based on Logistic Mapcited by 11

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.