# Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network

arXiv: https://arxiv.org/abs/2410.16159 · HTML: https://mlsanity.com/paper/2410.16159/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks](https://mlsanity.com/paper/1711.06798/) (2017)
- [AdderNet: Do We Really Need Multiplications in Deep Learning?](https://mlsanity.com/paper/1912.13200/) (2019)
- [Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks](https://mlsanity.com/paper/1905.10952/) (2019)
- [Input Invex Neural Network](https://mlsanity.com/paper/2106.08748/) (2021)
- [Learning Invariances with Generalised Input-Convex Neural Networks](https://mlsanity.com/paper/2204.07009/) (2022)
- [Noisy Heuristics NAS: A Network Morphism based Neural Architecture Search using Heuristics](https://mlsanity.com/paper/2207.04467/) (2022)
- [EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models](https://mlsanity.com/paper/2212.11803/) (2022)

## Abstract (arXiv)

Artificial Neural Networks of varying architectures are generally paired with affine transformation at the core. However, we find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance (as used in Radial Basis Function Network). In this work, we explore the generalization of dot product neurons to lp-norm, metrics, and beyond. We find that metrics as transform performs similarly to affine transform when used in MultiLayer Perceptron or Convolutional Neural Network. Moreover, we explore various properties of Metrics, compare it with Affine, and present multiple cases where metrics seem to provide better interpretability. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.