# Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets

arXiv: https://arxiv.org/abs/2206.09697 · HTML: https://mlsanity.com/paper/2206.09697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 20 Jun 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [PupilNet: Convolutional Neural Networks for Robust Pupil Detection](https://mlsanity.com/paper/1601.04902/) (2016)
- [ElSe: Ellipse Selection for Robust Pupil Detection in Real-World Environments](https://mlsanity.com/paper/1511.06575/) (2015)
- [Fully Convolutional Neural Networks for Raw Eye Tracking Data Segmentation, Generation, and Reconstruction](https://mlsanity.com/paper/2002.10905/) (2020)
- [Rotated Ring, Radial and Depth Wise Separable Radial Convolutions](https://mlsanity.com/paper/2010.00873/) (2020)
- [Eye movement velocity and gaze data generator for evaluation, robustness testing and assess of eye tracking software and visualization tools](https://mlsanity.com/paper/1808.09296/) (2018)
- [Learning to Validate the Quality of Detected Landmarks](https://mlsanity.com/paper/1901.10143/) (2019)
- [Weight and Gradient Centralization in Deep Neural Networks](https://mlsanity.com/paper/2010.00866/) (2020)
- [Fast camera focus estimation for gaze-based focus control](https://mlsanity.com/paper/1711.03306/) (2017)
- [Eye movement simulation and detector creation to reduce laborious parameter adjustments](https://mlsanity.com/paper/1804.00970/) (2018)
- [Training Decision Trees as Replacement for Convolution Layers](https://mlsanity.com/paper/1905.10073/) (2019)
- [The Gaze and Mouse Signal as additional Source for User Fingerprints in Browser Applications](https://mlsanity.com/paper/2101.03793/) (2021)
- [Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gaze](https://mlsanity.com/paper/2201.06799/) (2022)
- [Multi Layer Neural Networks as Replacement for Pooling Operations](https://mlsanity.com/paper/2006.06969/) (2020)
- [Explainable Online Validation of Machine Learning Models for Practical Applications](https://mlsanity.com/paper/2010.00821/) (2020)
- [A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis](https://mlsanity.com/paper/2101.04318/) (2021)
- [1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning](https://mlsanity.com/paper/2102.01921/) (2021)
- [HPCGen: Hierarchical K-Means Clustering and Level Based Principal Components for Scan Path Genaration](https://mlsanity.com/paper/2201.08354/) (2022)
- [Maximum and Leaky Maximum Propagation](https://mlsanity.com/paper/2105.10277/) (2021)
- [Tensor Normalization and Full Distribution Training](https://mlsanity.com/paper/2109.02345/) (2021)
- [GroupGazer: A Tool to Compute the Gaze per Participant in Groups with integrated Calibration to Map the Gaze Online to a Screen or Beamer Projection](https://mlsanity.com/paper/2201.07692/) (2022)

## Abstract (arXiv)

In this report, we combine the idea of Wide ResNets and transfer learning to optimize the architecture of deep neural networks. The first improvement of the architecture is the use of all layers as information source for the last layer. This idea comes from transfer learning, which uses networks pre-trained on other data and extracts different levels of the network as input for the new task. The second improvement is the use of deeper layers instead of deeper sequences of blocks. This idea comes from Wide ResNets. Using both optimizations, both high data augmentation and standard data augmentation can produce better results for different models. Link: https://github.com/wolfgangfuhl/PublicationStuff/tree/master/TechnicalReport1/Supp