# Second-order Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1703.06817 · HTML: https://mlsanity.com/paper/1703.06817/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2017.

## Built on by (most cited first, 7 of 7)

- [Robust Differentiable SVD](https://mlsanity.com/paper/2104.03821/) (2021)
- [Deep Riemannian Networks for End-to-End EEG Decoding](https://mlsanity.com/paper/2212.10426/) (2022)
- [Deep Efficient Continuous Manifold Learning for Time Series Modeling](https://mlsanity.com/paper/2112.03379/) (2021)
- [Differentially Private Fr'echet Mean on the Manifold of Symmetric Positive Definite (SPD) Matrices with log-Euclidean Metric](https://mlsanity.com/paper/2208.04245/) (2022)
- [Convolution goes higher-order: a biologically inspired mechanism empowers image classification](https://mlsanity.com/paper/2412.06740/) (2024)
- [Near out-of-distribution detection for low-resolution radar micro-Doppler signatures](https://mlsanity.com/paper/2205.07869/) (2022)
- [A geometrically aware auto-encoder for multi-texture synthesis](https://mlsanity.com/paper/2302.01616/) (2023)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have been successfully applied to many computer vision tasks, such as image classification. By performing linear combinations and element-wise nonlinear operations, these networks can be thought of as extracting solely first-order information from an input image. In the past, however, second-order statistics computed from handcrafted features, e.g., covariances, have proven highly effective in diverse recognition tasks. In this paper, we introduce a novel class of CNNs that exploit second-order statistics. To this end, we design a series of new layers that (i) extract a covariance matrix from convolutional activations, (ii) compute a parametric, second-order transformation of a matrix, and (iii) perform a parametric vectorization of a matrix. These operations can be assembled to form a Covariance Descriptor Unit (CDU), which replaces the fully-connected layers of standard CNNs. Our experiments demonstrate the benefits of our new architecture, which outperform the first-order CNNs, while relying on up to 90% fewer parameters.