mlsanityData to 7 Oct 2026

Tensor Normalization and Full Distribution Training

arXiv 2109.02345 ↗First posted 6 Sep 2021cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In this work, we introduce pixel wise tensor normalization, which is inserted after rectifier linear units and, together with batch normalization, provides a significant improvement in the accuracy of modern deep neural networks. In addition, this work deals with the robustness of networks. We show that the factorized superposition of images from the training set and the reformulation of the multi class problem into a multi-label problem yields significantly more robust networks. The reformulation and the adjustment of the multi class log loss also improves the results compared to the overlay with only one class as label. https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/?p=%2FTNandFDT&mode=list

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 66 of its 112 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

66 of 112 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2016Layer Normalizationcited by 5,579
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  11. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  12. 2016Wide Residual Networkscited by 3,057
38 more
  1. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  4. 2016Adversarial examples in the physical worldcited by 1,762
  5. 2018Group Normalizationcited by 1,514
  6. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  7. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  8. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  9. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  10. 2017Residual Attention Network for Image Classificationcited by 385
  11. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  12. 2016CNN-RNN: A Unified Framework for Multi-label Image Classificationcited by 189
  13. 2016Deep Pyramidal Residual Networkscited by 163
  14. 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
  15. 2017Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classificationcited by 85
  16. 2020Gradient Centralization: A New Optimization Technique for Deep Neural Networkscited by 82
  17. 2017CNN-Based Projected Gradient Descent for Consistent Image Reconstructioncited by 75
  18. 2014CNN: Single-label to Multi-labelcited by 62
  19. 2013Deep Convolutional Ranking for Multilabel Image Annotationcited by 47
  20. 2019Micro-Batch Training with Batch-Channel Normalization and Weight Standardizationcited by 45
  21. 2015Simultaneous Feature Learning and Hash Coding with Deep Neural Networkscited by 44
  22. 2015Loss Functions for Top-k Error: Analysis and Insightscited by 38
  23. 2017Riemannian approach to batch normalizationcited by 35
  24. 2017Adversarial Example Defenses: Ensembles of Weak Defenses are not Strongcited by 32
  25. 2021TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Typescited by 32
  26. 2017Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networkscited by 31
  27. 2015Top-k Multiclass SVMcited by 28
  28. 2017Attacking Binarized Neural Networkscited by 28
  29. 2020EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attackscited by 28
  30. 2015Deep Semantic Ranking Based Hashing for Multi-Label Image Retrievalcited by 24
  31. 2016Multi-Label Image Classification with Regional Latent Semantic Dependenciescited by 23
  32. 2016PupilNet: Convolutional Neural Networks for Robust Pupil Detectioncited by 20
  33. 2016Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classificationcited by 20
  34. 2018Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functionscited by 17
  35. 2015ElSe: Ellipse Selection for Robust Pupil Detection in Real-World Environmentscited by 16
  36. 2017PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detectioncited by 15
  37. 2020Fully Convolutional Neural Networks for Raw Eye Tracking Data Segmentation, Generation, and Reconstructioncited by 12
  38. 2020Rotated Ring, Radial and Depth Wise Separable Radial Convolutionscited by 12

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3.

3 papers.

  1. 2022Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gazecited by 9
  2. 2022GroupGazer: A Tool to Compute the Gaze per Participant in Groups with integrated Calibration to Map the Gaze Online to a Screen or Beamer Projectioncited by 2
  3. 2022Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnetscited by 0