# Jitter: Random Jittering Loss Function

arXiv: https://arxiv.org/abs/2106.13749 · HTML: https://mlsanity.com/paper/2106.13749/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Adversarial AutoAugment](https://mlsanity.com/paper/1912.11188/) (2019)
- [Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate](https://mlsanity.com/paper/1806.05161/) (2018)
- [Delving Deep into Label Smoothing](https://mlsanity.com/paper/2011.12562/) (2020)
- [Do We Need Zero Training Loss After Achieving Zero Training Error?](https://mlsanity.com/paper/2002.08709/) (2020)
- [AutoDropout: Learning Dropout Patterns to Regularize Deep Networks](https://mlsanity.com/paper/2101.01761/) (2021)
- [Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data](https://mlsanity.com/paper/1909.09148/) (2019)
- [Towards Understanding Label Smoothing](https://mlsanity.com/paper/2006.11653/) (2020)
- [Batch Group Normalization](https://mlsanity.com/paper/2012.02782/) (2020)
- [Data Augmentation for Object Detection via Differentiable Neural Rendering](https://mlsanity.com/paper/2103.02852/) (2021)

## Built on by (most cited first, 3 of 3)

- [Learn to Enhance the Negative Information in Convolutional Neural Network](https://mlsanity.com/paper/2306.10536/) (2023)
- [NSTR: Neural Spectral Transport Representation for Space-Varying Frequency Fields](https://mlsanity.com/paper/2511.18384/) (2025)
- [Scaling Implicit Fields via Hypernetwork-Driven Multiscale Coordinate Transformations](https://mlsanity.com/paper/2511.18387/) (2025)

## Abstract (arXiv)

Regularization plays a vital role in machine learning optimization. One novel regularization method called flooding makes the training loss fluctuate around the flooding level. It intends to make the model continue to random walk until it comes to a flat loss landscape to enhance generalization. However, the hyper-parameter flooding level of the flooding method fails to be selected properly and uniformly. We propose a novel method called Jitter to improve it. Jitter is essentially a kind of random loss function. Before training, we randomly sample the Jitter Point from a specific probability distribution. The flooding level should be replaced by Jitter point to obtain a new target function and train the model accordingly. As Jitter point acting as a random factor, we actually add some randomness to the loss function, which is consistent with the fact that there exists innumerable random behaviors in the learning process of the machine learning model and is supposed to make the model more robust. In addition, Jitter performs random walk randomly which divides the loss curve into small intervals and then flipping them over, ideally making the loss curve much flatter and enhancing generalization ability. Moreover, Jitter can be a domain-, task-, and model-independent regularization method and train the model effectively after the training error reduces to zero. Our experimental results show that Jitter method can improve model performance more significantly than the previous flooding method and make the test loss curve descend twice.