# Grassmannian learning mutual subspace method for image set recognition

arXiv: https://arxiv.org/abs/2111.04352 · HTML: https://mlsanity.com/paper/2111.04352/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 90 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Building Deep Networks on Grassmann Manifolds](https://mlsanity.com/paper/1611.05742/) (2016)
- [Temperature check: theory and practice for training models with softmax-cross-entropy losses](https://mlsanity.com/paper/2010.07344/) (2020)
- [Training Deep Networks with Structured Layers by Matrix Backpropagation](https://mlsanity.com/paper/1509.07838/) (2015)
- [From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices](https://mlsanity.com/paper/1407.1120/) (2014)
- [Dictionary Learning and Sparse Coding on Grassmann Manifolds: An Extrinsic Solution](https://mlsanity.com/paper/1310.4891/) (2013)
- [CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces](https://mlsanity.com/paper/1805.07621/) (2018)
- [Image Set based Collaborative Representation for Face Recognition](https://mlsanity.com/paper/1308.6687/) (2013)
- [Efficient Image Set Classification using Linear Regression based Image Reconstruction](https://mlsanity.com/paper/1701.02485/) (2017)
- [Constrained Mutual Convex Cone Method for Image Set Based Recognition](https://mlsanity.com/paper/1903.06549/) (2019)
- [ADMM-based Decoder for Binary Linear Codes Aided by Deep Learning](https://mlsanity.com/paper/2002.07601/) (2020)

## Built on by (most cited first, 3 of 3)

- [Grassmann Manifold Flows for Stable Shape Generation](https://mlsanity.com/paper/2211.02900/) (2022)
- [Interact with me: Joint Egocentric Forecasting of Intent to Interact, Attitude and Social Actions](https://mlsanity.com/paper/2412.16698/) (2024)
- [Color Image Set Recognition Based on Quaternionic Grassmannians](https://mlsanity.com/paper/2505.23629/) (2025)

## Abstract (arXiv)

This paper addresses the problem of object recognition given a set of images as input (e.g., multiple camera sources and video frames). Convolutional neural network (CNN)-based frameworks do not exploit these sets effectively, processing a pattern as observed, not capturing the underlying feature distribution as it does not consider the variance of images in the set. To address this issue, we propose the Grassmannian learning mutual subspace method (G-LMSM), a NN layer embedded on top of CNNs as a classifier, that can process image sets more effectively and can be trained in an end-to-end manner. The image set is represented by a low-dimensional input subspace; and this input subspace is matched with reference subspaces by a similarity of their canonical angles, an interpretable and easy to compute metric. The key idea of G-LMSM is that the reference subspaces are learned as points on the Grassmann manifold, optimized with Riemannian stochastic gradient descent. This learning is stable, efficient and theoretically well-grounded. We demonstrate the effectiveness of our proposed method on hand shape recognition, face identification, and facial emotion recognition.