# Complement Objective Training

arXiv: https://arxiv.org/abs/1903.01182 · HTML: https://mlsanity.com/paper/1903.01182/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 15 of 15)

- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [From Big to Small: Adaptive Learning to Partial-Set Domains](https://mlsanity.com/paper/2203.07375/) (2022)
- [A Generative Approach for Script Event Prediction via Contrastive Fine-tuning](https://mlsanity.com/paper/2212.03496/) (2022)
- [G2Pxy: Generative Open-Set Node Classification on Graphs with Proxy Unknowns](https://mlsanity.com/paper/2308.05463/) (2023)
- [Does Adversarial Oversampling Help us?](https://mlsanity.com/paper/2108.10697/) (2021)
- [From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue](https://mlsanity.com/paper/2308.12648/) (2023)
- [Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks](https://mlsanity.com/paper/2101.03057/) (2021)
- [A Softmax-free Loss Function Based on Predefined Optimal-distribution of Latent Features for Deep Learning Classifier](https://mlsanity.com/paper/2111.15449/) (2021)
- [Few-shot 1/a Anomalies Feedback : Damage Vision Mining Opportunity and Embedding Feature Imbalance](https://mlsanity.com/paper/2307.12676/) (2023)
- [Partial Distribution Matching via Partial Wasserstein Adversarial Networks](https://mlsanity.com/paper/2409.10499/) (2024)
- [Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification](https://mlsanity.com/paper/2101.05624/) (2021)
- [Mixing between the Cross Entropy and the Expectation Loss Terms](https://mlsanity.com/paper/2109.05635/) (2021)
- [PoliTO-IIT-CINI Submission to the EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition](https://mlsanity.com/paper/2209.04525/) (2022)
- [A Robust Negative Learning Approach to Partial Domain Adaptation Using Source Prototypes](https://mlsanity.com/paper/2309.03531/) (2023)
- [Robust Class-Conditional Distribution Alignment for Partial Domain Adaptation](https://mlsanity.com/paper/2310.12060/) (2023)

## Abstract (arXiv)

Learning with a primary objective, such as softmax cross entropy for classification and sequence generation, has been the norm for training deep neural networks for years. Although being a widely-adopted approach, using cross entropy as the primary objective exploits mostly the information from the ground-truth class for maximizing data likelihood, and largely ignores information from the complement (incorrect) classes. We argue that, in addition to the primary objective, training also using a complement objective that leverages information from the complement classes can be effective in improving model performance. This motivates us to study a new training paradigm that maximizes the likelihood of the groundtruth class while neutralizing the probabilities of the complement classes. We conduct extensive experiments on multiple tasks ranging from computer vision to natural language understanding. The experimental results confirm that, compared to the conventional training with just one primary objective, training also with the complement objective further improves the performance of the state-of-the-art models across all tasks. In addition to the accuracy improvement, we also show that models trained with both primary and complement objectives are more robust to single-step adversarial attacks.