mlsanityData to 7 Oct 2026

Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network

arXiv 2410.16159 ↗First posted 21 Oct 2024cs.LG cs.CV cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial Neural Networks of varying architectures are generally paired with affine transformation at the core. However, we find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance (as used in Radial Basis Function Network). In this work, we explore the generalization of dot product neurons to lp-norm, metrics, and beyond. We find that metrics as transform performs similarly to affine transform when used in MultiLayer Perceptron or Convolutional Neural Network. Moreover, we explore various properties of Metrics, compare it with Affine, and present multiple cases where metrics seem to provide better interpretability. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

19 of 37 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2016Layer Normalizationcited by 5,579
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  8. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  10. 2016Input Convex Neural Networkscited by 407
  11. 2015Highway Networkscited by 257
  12. 2020The Hardware Lotterycited by 160
7 more
  1. 2017MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networkscited by 92
  2. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  3. 2019Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networkscited by 9
  4. 2021Input Invex Neural Networkcited by 4
  5. 2022Learning Invariances with Generalised Input-Convex Neural Networkscited by 3
  6. 2022Noisy Heuristics NAS: A Network Morphism based Neural Architecture Search using Heuristicscited by 2
  7. 2022EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Modelscited by 1

Built on by

Not cited yet.