# Hierarchical Novelty Detection for Visual Object Recognition

arXiv: https://arxiv.org/abs/1804.00722 · HTML: https://mlsanity.com/paper/1804.00722/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2018.

## Built on by (most cited first, 26 of 26)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Continuous Learning for Android Malware Detection](https://mlsanity.com/paper/2302.04332/) (2023)
- [Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Data](https://mlsanity.com/paper/2412.06284/) (2024)
- [Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity](https://mlsanity.com/paper/2303.05689/) (2023)
- [ProTeCt: Prompt Tuning for Taxonomic Open Set Classification](https://mlsanity.com/paper/2306.02240/) (2023)
- [Uncertainty-Guided Appearance-Motion Association Network for Out-of-Distribution Action Detection](https://mlsanity.com/paper/2409.09953/) (2024)
- [Semi-Supervised Learning with Taxonomic Labels](https://mlsanity.com/paper/2111.11595/) (2021)
- [Taxonomy-Aware Evaluation of Vision-Language Models](https://mlsanity.com/paper/2504.05457/) (2025)
- [Adaptive Hierarchical Graph Cut for Multi-granularity Out-of-distribution Detection](https://mlsanity.com/paper/2412.15668/) (2024)
- [Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detection](https://mlsanity.com/paper/2506.17633/) (2025)
- [Confident AI](https://mlsanity.com/paper/2202.05957/) (2022)
- [Differential Analysis of Triggers and Benign Features for Black-Box DNN Backdoor Detection](https://mlsanity.com/paper/2307.05422/) (2023)
- [Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language](https://mlsanity.com/paper/2605.29812/) (2026)
- [Embedding Semantic Hierarchy in Discrete Optimal Transport for Risk Minimization](https://mlsanity.com/paper/2105.00101/) (2021)
- [United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks](https://mlsanity.com/paper/2107.13627/) (2021)
- [Egocentric Hierarchical Visual Semantics](https://mlsanity.com/paper/2305.05422/) (2023)
- [Discriminative-Generative Representation Learning for One-Class Anomaly Detection](https://mlsanity.com/paper/2107.12753/) (2021)
- [Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness](https://mlsanity.com/paper/2605.30745/) (2026)
- [Exploring Adversarial Examples for Efficient Active Learning in Machine Learning Classifiers](https://mlsanity.com/paper/2109.10770/) (2021)
- [Rethinking Open-Set Object Detection: Issues, a New Formulation, and Taxonomy](https://mlsanity.com/paper/2207.09775/) (2022)
- [Collaborative Knowledge Distillation via a Learning-by-Education Node Community](https://mlsanity.com/paper/2410.00074/) (2024)
- [Balanced Hyperbolic Embeddings Are Natural Out-of-Distribution Detectors](https://mlsanity.com/paper/2506.10146/) (2025)
- [Bottom-up Hierarchical Classification Using Confusion-based Logit Compression](https://mlsanity.com/paper/2110.01756/) (2021)
- [Beyond Flat Unknown Labels in Open-World Object Detection](https://mlsanity.com/paper/2510.09173/) (2025)

## Abstract (arXiv)

Deep neural networks have achieved impressive success in large-scale visual object recognition tasks with a predefined set of classes. However, recognizing objects of novel classes unseen during training still remains challenging. The problem of detecting such novel classes has been addressed in the literature, but most prior works have focused on providing simple binary or regressive decisions, e.g., the output would be "known," "novel," or corresponding confidence intervals. In this paper, we study more informative novelty detection schemes based on a hierarchical classification framework. For an object of a novel class, we aim for finding its closest super class in the hierarchical taxonomy of known classes. To this end, we propose two different approaches termed top-down and flatten methods, and their combination as well. The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy. Furthermore, our method can generate a hierarchical embedding that leads to improved generalized zero-shot learning performance in combination with other commonly-used semantic embeddings.