# A Meta-learning Formulation of the Autoencoder Problem for Non-linear Dimensionality Reduction

arXiv: https://arxiv.org/abs/2207.06676 · HTML: https://mlsanity.com/paper/2207.06676/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://mlsanity.com/paper/1904.12760/) (2019)
- [Bilevel Optimization: Convergence Analysis and Enhanced Design](https://mlsanity.com/paper/2010.07962/) (2020)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems](https://mlsanity.com/paper/1808.01346/) (2018)
- [A Multifidelity Ensemble Kalman Filter with Reduced Order Control Variates](https://mlsanity.com/paper/2007.00793/) (2020)
- [Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Physics-Informed Autoencoders](https://mlsanity.com/paper/2102.13025/) (2021)
- [Complexity guarantees for an implicit smoothing-enabled method for stochastic MPECs](https://mlsanity.com/paper/2104.08406/) (2021)
- [An Optimization Framework to Improve 4D-Var Data Assimilation System Performance](https://mlsanity.com/paper/1308.1860/) (2013)
- [Investigation of Nonlinear Model Order Reduction of the Quasigeostrophic Equations through a Physics-Informed Convolutional Autoencoder](https://mlsanity.com/paper/2108.12344/) (2021)

## Built on by (most cited first, 1 of 1)

- [Small-data Reduced Order Modeling of Chaotic Dynamics through SyCo-AE: Synthetically Constrained Autoencoders](https://mlsanity.com/paper/2305.08036/) (2023)

## Abstract (arXiv)

A rapidly growing area of research is the use of machine learning approaches such as autoencoders for dimensionality reduction of data and models in scientific applications. We show that the canonical formulation of autoencoders suffers from several deficiencies that can hinder their performance. Using a meta-learning approach, we reformulate the autoencoder problem as a bi-level optimization procedure that explicitly solves the dimensionality reduction task. We prove that the new formulation corrects the identified deficiencies with canonical autoencoders, provide a practical way to solve it, and showcase the strength of this formulation with a simple numerical illustration.