# TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1604.06318 · HTML: https://mlsanity.com/paper/1604.06318/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2016.

## Built on by (most cited first, 30 of 67)

- [Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning](https://mlsanity.com/paper/2103.01315/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Transformation-Equivariant 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2211.11962/) (2022)
- [BEVPlace: Learning LiDAR-based Place Recognition using Bird's Eye View Images](https://mlsanity.com/paper/2302.14325/) (2023)
- [Deep Learning for UAV-based Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2110.12638/) (2021)
- [RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning](https://mlsanity.com/paper/2202.13094/) (2022)
- [Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions](https://mlsanity.com/paper/2107.14519/) (2021)
- [Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities](https://mlsanity.com/paper/2206.09880/) (2022)
- [RIC-CNN: Rotation-Invariant Coordinate Convolutional Neural Network](https://mlsanity.com/paper/2211.11812/) (2022)
- [Equivariance Discovery by Learned Parameter-Sharing](https://mlsanity.com/paper/2204.03640/) (2022)
- [A PAC-Bayesian Generalization Bound for Equivariant Networks](https://mlsanity.com/paper/2210.13150/) (2022)
- [Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales](https://mlsanity.com/paper/2106.06418/) (2021)
- [Revisiting Data Augmentation for Rotational Invariance in Convolutional Neural Networks](https://mlsanity.com/paper/2310.08429/) (2023)
- [Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function](https://mlsanity.com/paper/2110.00899/) (2021)
- [Training or Architecture? How to Incorporate Invariance in Neural Networks](https://mlsanity.com/paper/2106.10044/) (2021)
- [Robust Perception through Equivariance](https://mlsanity.com/paper/2212.06079/) (2022)
- [Steerable Transformers for Volumetric Data](https://mlsanity.com/paper/2405.15932/) (2024)
- [SPIN: Simplifying Polar Invariance for Neural networks Application to vision-based irradiance forecasting](https://mlsanity.com/paper/2111.14507/) (2021)
- [Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration](https://mlsanity.com/paper/2202.03967/) (2022)
- [Equivariant Wavelets: Fast Rotation and Translation Invariant Wavelet Scattering Transforms](https://mlsanity.com/paper/2104.11244/) (2021)
- [Mean Embeddings with Test-Time Data Augmentation for Ensembling of Representations](https://mlsanity.com/paper/2106.08038/) (2021)
- [Sampling Equivariant Self-attention Networks for Object Detection in Aerial Images](https://mlsanity.com/paper/2111.03420/) (2021)
- [Generating Unrestricted Adversarial Examples via Three Parameters](https://mlsanity.com/paper/2103.07640/) (2021)
- [Implicit Equivariance in Convolutional Networks](https://mlsanity.com/paper/2111.14157/) (2021)
- [Steerable Equivariant Representation Learning](https://mlsanity.com/paper/2302.11349/) (2023)
- [A Regularization-Guided Equivariant Approach for Image Restoration](https://mlsanity.com/paper/2505.19799/) (2025)
- [Rotation Equivariant Arbitrary-scale Image Super-Resolution](https://mlsanity.com/paper/2508.05160/) (2025)
- [Towards Invariance to Node Identifiers in Graph Neural Networks](https://mlsanity.com/paper/2502.13660/) (2025)
- [Spectral State Space Model for Rotation-Invariant Visual Representation Learning](https://mlsanity.com/paper/2503.06369/) (2025)
- [Target Detection and Segmentation in Circular-Scan Synthetic-Aperture-Sonar Images using Semi-Supervised Convolutional Encoder-Decoders](https://mlsanity.com/paper/2101.03603/) (2021)

## Abstract (arXiv)

In this paper we present a deep neural network topology that incorporates a simple to implement transformation invariant pooling operator (TI-POOLING). This operator is able to efficiently handle prior knowledge on nuisance variations in the data, such as rotation or scale changes. Most current methods usually make use of dataset augmentation to address this issue, but this requires larger number of model parameters and more training data, and results in significantly increased training time and larger chance of under- or overfitting. The main reason for these drawbacks is that the learned model needs to capture adequate features for all the possible transformations of the input. On the other hand, we formulate features in convolutional neural networks to be transformation-invariant. We achieve that using parallel siamese architectures for the considered transformation set and applying the TI-POOLING operator on their outputs before the fully-connected layers. We show that this topology internally finds the most optimal "canonical" instance of the input image for training and therefore limits the redundancy in learned features. This more efficient use of training data results in better performance on popular benchmark datasets with smaller number of parameters when comparing to standard convolutional neural networks with dataset augmentation and to other baselines.