mlsanityData to 7 Oct 2026

DeepPermNet: Visual Permutation Learning

arXiv 1704.02729 ↗First posted 10 Apr 2017cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

We present a principled approach to uncover the structure of visual data by solving a novel deep learning task coined visual permutation learning. The goal of this task is to find the permutation that recovers the structure of data from shuffled versions of it. In the case of natural images, this task boils down to recovering the original image from patches shuffled by an unknown permutation matrix. Unfortunately, permutation matrices are discrete, thereby posing difficulties for gradient-based methods. To this end, we resort to a continuous approximation of these matrices using doubly-stochastic matrices which we generate from standard CNN predictions using Sinkhorn iterations. Unrolling these iterations in a Sinkhorn network layer, we propose DeepPermNet, an end-to-end CNN model for this task. The utility of DeepPermNet is demonstrated on two challenging computer vision problems, namely, (i) relative attributes learning and (ii) self-supervised representation learning. Our results show state-of-the-art performance on the Public Figures and OSR benchmarks for (i) and on the classification and segmentation tasks on the PASCAL VOC dataset for (ii).

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 8; 2023: 6; 2024: 4; 2025: 3; 2026: 1.

32 papers.

Sort:

By area:

  1. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  2. 2022Tutorial on amortized optimizationcited by 83
  3. 2022A Generalist Neural Algorithmic Learnercited by 58
  4. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
  5. 2021Geographical Knowledge-driven Representation Learning for Remote Sensing Imagescited by 37
  6. 2022A Unified Framework for Implicit Sinkhorn Differentiationcited by 28
  7. 2022Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformercited by 21
  8. 2022Location-Aware Self-Supervised Transformers for Semantic Segmentationcited by 20
  9. 2023DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positionscited by 19
  10. 2021SPANet: Generalized Permutationless Set Assignment for Particle Physics using Symmetry Preserving Attentioncited by 18
  11. 2021JigsawGAN: Auxiliary Learning for Solving Jigsaw Puzzles with Generative Adversarial Networkscited by 16
  12. 2024LinSATNet: The Positive Linear Satisfiability Neural Networkscited by 16
20 more
  1. 2021Self-Supervised Learning of Audio Representations from Permutations with Differentiable Rankingcited by 14
  2. 2021Robust Partial-to-Partial Point Cloud Registration in a Full Rangecited by 8
  3. 2023DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Datacited by 8
  4. 2021Contrastive Learning for Unsupervised Image-to-Image Translationcited by 7
  5. 2021Boosting Generative Zero-Shot Learning by Synthesizing Diverse Features with Attribute Augmentationcited by 7
  6. 2024GLinSAT: The General Linear Satisfiability Neural Network Layer By Accelerated Gradient Descentcited by 7
  7. 2022Contextual Text Block Detection towards Scene Text Understandingcited by 6
  8. 2024Manual-PA: Learning 3D Part Assembly from Instruction Diagramscited by 6
  9. 2021Unlabeled Principal Component Analysis and Matrix Completioncited by 5
  10. 2023Kissing to Find a Match: Efficient Low-Rank Permutation Representationcited by 5
  11. 2024SymmetricDiffusers: Learning Discrete Diffusion on Finite Symmetric Groupscited by 5
  12. 2021Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignmentcited by 4
  13. 2021Low-rank Matrix Recovery With Unknown Correspondencecited by 4
  14. 2023DISGO: Automatic End-to-End Evaluation for Scene Text OCRcited by 4
  15. 2023Differentiable Random Partition Modelscited by 3
  16. 2022Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Predictioncited by 2
  17. 2023Self-Ordering Point Cloudscited by 2
  18. 2025Zero-Shot Learning with Subsequence Reordering Pretraining for Compound-Protein Interactioncited by 2
  19. 2025SPAR: Self-supervised Placement-Aware Representation Learning for Distributed Sensingcited by 0
  20. 2026Learning Permutation from Structure Without Supervisioncited by 0