# Learning Unitary Operators with Help From u(n)

arXiv: https://arxiv.org/abs/1607.04903 · HTML: https://mlsanity.com/paper/1607.04903/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2016.

## Built on by (most cited first, 15 of 15)

- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks](https://mlsanity.com/paper/2104.05097/) (2021)
- [projUNN: efficient method for training deep networks with unitary matrices](https://mlsanity.com/paper/2203.05483/) (2022)
- [A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold](https://mlsanity.com/paper/2302.08210/) (2023)
- [Group and Shuffle: Efficient Structured Orthogonal Parametrization](https://mlsanity.com/paper/2406.10019/) (2024)
- [Optimal quantum dataset for learning a unitary transformation](https://mlsanity.com/paper/2203.00546/) (2022)
- [Neural Gaussian Scale-Space Fields](https://mlsanity.com/paper/2405.20980/) (2024)
- [Coordinate descent on the orthogonal group for recurrent neural network training](https://mlsanity.com/paper/2108.00051/) (2021)
- [Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs](https://mlsanity.com/paper/2205.08385/) (2022)
- [Orthogonal Gated Recurrent Unit with Neumann-Cayley Transformation](https://mlsanity.com/paper/2208.06496/) (2022)
- [Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax](https://mlsanity.com/paper/2208.05719/) (2022)
- [Superstate Quantum Mechanics](https://mlsanity.com/paper/2502.00037/) (2025)
- [Variational Optimization for Quantum Problems using Deep Generative Networks](https://mlsanity.com/paper/2404.18041/) (2024)
- [Operator Learning for Schr"odinger Equation: Unitarity, Error Bounds, and Time Generalization](https://mlsanity.com/paper/2505.18288/) (2025)
- [Semidefinite Programming for Quantum Channel Learning](https://mlsanity.com/paper/2601.12502/) (2026)

## Abstract (arXiv)

A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem. The use of a norm-preserving transition operator can address this issue, but parametrization is challenging. In this work we focus on unitary operators and describe a parametrization using the Lie algebra mathfraku(n) associated with the Lie group U(n) of n × n unitary matrices. The exponential map provides a correspondence between these spaces, and allows us to define a unitary matrix using n2 real coefficients relative to a basis of the Lie algebra. The parametrization is closed under additive updates of these coefficients, and thus provides a simple space in which to do gradient descent. We demonstrate the effectiveness of this parametrization on the problem of learning arbitrary unitary operators, comparing to several baselines and outperforming a recently-proposed lower-dimensional parametrization. We additionally use our parametrization to generalize a recently-proposed unitary recurrent neural network to arbitrary unitary matrices, using it to solve standard long-memory tasks.