# Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition

arXiv: https://arxiv.org/abs/1908.01174 · HTML: https://mlsanity.com/paper/1908.01174/

## Key facts

- Cited by 10 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [Cluster and Aggregate: Face Recognition with Large Probe Set](https://mlsanity.com/paper/2210.10864/) (2022)
- [Energy-constrained Self-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2101.00316/) (2021)
- [Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology](https://mlsanity.com/paper/2101.06775/) (2021)
- [Identity-aware Facial Expression Recognition in Compressed Video](https://mlsanity.com/paper/2101.00317/) (2021)
- [Harnessing Unrecognizable Faces for Improving Face Recognition](https://mlsanity.com/paper/2106.04112/) (2021)
- [CoNAN: Conditional Neural Aggregation Network For Unconstrained Face Feature Fusion](https://mlsanity.com/paper/2307.10237/) (2023)
- [Interpreting Depression From Question-wise Long-term Video Recording of SDS Evaluation](https://mlsanity.com/paper/2106.13393/) (2021)
- [FaceCoresetNet: Differentiable Coresets for Face Set Recognition](https://mlsanity.com/paper/2308.14075/) (2023)
- [Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning](https://mlsanity.com/paper/2408.03404/) (2024)
- [DCSCR: A Class-Specific Collaborative Representation based Network for Image Set Classification](https://mlsanity.com/paper/2508.12745/) (2025)

## Abstract (arXiv)

We consider the problem of comparing the similarity of image sets with variable-quantity, quality and un-ordered heterogeneous images. We use feature restructuring to exploit the correlations of both inner&inter-set images. Specifically, the residual self-attention can effectively restructure the features using the other features within a set to emphasize the discriminative images and eliminate the redundancy. Then, a sparse/collaborative learning-based dependency-guided representation scheme reconstructs the probe features conditional to the gallery features in order to adaptively align the two sets. This enables our framework to be compatible with both verification and open-set identification. We show that the parametric self-attention network and non-parametric dictionary learning can be trained end-to-end by a unified alternative optimization scheme, and that the full framework is permutation-invariant. In the numerical experiments we conducted, our method achieves top performance on competitive image set/video-based face recognition and person re-identification benchmarks.