# Fast Approximation of Rotations and Hessians matrices

arXiv: https://arxiv.org/abs/1404.7195 · HTML: https://mlsanity.com/paper/1404.7195/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2014.

## Built on by (most cited first, 8 of 8)

- [Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models](https://mlsanity.com/paper/2112.00029/) (2021)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Learning quantum data with the quantum Earth Mover's distance](https://mlsanity.com/paper/2101.03037/) (2021)
- [Efficient Identification of Butterfly Sparse Matrix Factorizations](https://mlsanity.com/paper/2110.01230/) (2021)
- [Parallelized Computation and Backpropagation Under Angle-Parametrized Orthogonal Matrices](https://mlsanity.com/paper/2106.00003/) (2021)
- [Arithmetic Circuits, Structured Matrices and (not so) Deep Learning](https://mlsanity.com/paper/2206.12490/) (2022)
- [Acceleration Method for Learning Fine-Layered Optical Neural Networks](https://mlsanity.com/paper/2109.01731/) (2021)
- [Learning to Accelerate by the Methods of Step-size Planning](https://mlsanity.com/paper/2204.01705/) (2022)

## Abstract (arXiv)

A new method to represent and approximate rotation matrices is introduced. The method represents approximations of a rotation matrix Q with linearithmic complexity, i.e. with frac12nlg(n) rotations over pairs of coordinates, arranged in an FFT-like fashion. The approximation is "learned" using gradient descent. It allows to represent symmetric matrices H as QDQT where D is a diagonal matrix. It can be used to approximate covariance matrix of Gaussian models in order to speed up inference, or to estimate and track the inverse Hessian of an objective function by relating changes in parameters to changes in gradient along the trajectory followed by the optimization procedure. Experiments were conducted to approximate synthetic matrices, covariance matrices of real data, and Hessian matrices of objective functions involved in machine learning problems.