# Nuisance-Label Supervision: Robustness Improvement by Free Labels

arXiv: https://arxiv.org/abs/2110.07118 · HTML: https://mlsanity.com/paper/2110.07118/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 30 of 33)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition](https://mlsanity.com/paper/1912.05534/) (2019)
- [Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings](https://mlsanity.com/paper/1809.02169/) (2018)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery](https://mlsanity.com/paper/1711.09082/) (2017)
- [Synthesizing Training Images for Boosting Human 3D Pose Estimation](https://mlsanity.com/paper/1604.02703/) (2016)
- [Synthetic Humans for Action Recognition from Unseen Viewpoints](https://mlsanity.com/paper/1912.04070/) (2019)
- [MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild](https://mlsanity.com/paper/1607.02046/) (2016)
- [Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points](https://mlsanity.com/paper/1802.07898/) (2018)
- [Procedural Generation of Videos to Train Deep Action Recognition Networks](https://mlsanity.com/paper/1612.00881/) (2016)
- [A simple way to make neural networks robust against diverse image corruptions](https://mlsanity.com/paper/2001.06057/) (2020)
- [Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing](https://mlsanity.com/paper/1612.02699/) (2016)
- [R-CNNs for Pose Estimation and Action Detection](https://mlsanity.com/paper/1406.5212/) (2014)
- [Pose-conditioned Spatio-Temporal Attention for Human Action Recognition](https://mlsanity.com/paper/1703.10106/) (2017)

## Built on by (most cited first, 1 of 1)

- [Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation](https://mlsanity.com/paper/2511.12491/) (2025)

## Abstract (arXiv)

In this paper, we present a Nuisance-label Supervision (NLS) module, which can make models more robust to nuisance factor variations. Nuisance factors are those irrelevant to a task, and an ideal model should be invariant to them. For example, an activity recognition model should perform consistently regardless of the change of clothes and background. But our experiments show existing models are far from this capability. So we explicitly supervise a model with nuisance labels to make extracted features less dependent on nuisance factors. Although the values of nuisance factors are rarely annotated, we demonstrate that besides existing annotations, nuisance labels can be acquired freely from data augmentation and synthetic data. Experiments show consistent improvement in robustness towards image corruption and appearance change in action recognition.