# A semantics-driven methodology for high-quality image annotation

arXiv: https://arxiv.org/abs/2307.14119 · HTML: https://mlsanity.com/paper/2307.14119/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [From ImageNet to Image Classification: Contextualizing Progress on Benchmarks](https://mlsanity.com/paper/2005.11295/) (2020)
- [Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods](https://mlsanity.com/paper/1907.09358/) (2019)
- [Towards Visual Semantics](https://mlsanity.com/paper/2104.12379/) (2021)
- [Assessing Data Quality of Annotations with Krippendorff Alpha For Applications in Computer Vision](https://mlsanity.com/paper/1912.10107/) (2019)
- [krippendorffsalpha: An R Package for Measuring Agreement Using Krippendorff's Alpha Coefficient](https://mlsanity.com/paper/2103.12170/) (2021)
- [Aligning Visual and Lexical Semantics](https://mlsanity.com/paper/2212.06629/) (2022)
- [Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal](https://mlsanity.com/paper/2110.13504/) (2021)
- [Incremental Image Labeling via Iterative Refinement](https://mlsanity.com/paper/2304.08989/) (2023)
- [Egocentric Hierarchical Visual Semantics](https://mlsanity.com/paper/2305.05422/) (2023)

## Built on by (most cited first, 6 of 6)

- [Lexical Diversity in Kinship Across Languages and Dialects](https://mlsanity.com/paper/2308.13056/) (2023)
- [From Knowledge Representation to Knowledge Organization and Back](https://mlsanity.com/paper/2312.07302/) (2023)
- [From Knowledge Organization to Knowledge Representation and Back](https://mlsanity.com/paper/2401.11753/) (2024)
- [What can Computer Vision learn from Ranganathan?](https://mlsanity.com/paper/2601.22634/) (2026)
- [Crowdsourcing of Real-world Image Annotation via Visual Properties](https://mlsanity.com/paper/2604.14449/) (2026)
- [Zero-Shot Chinese Character Recognition via Global-Local Dual-Branch Alignment and Hierarchical Inference](https://mlsanity.com/paper/2605.08814/) (2026)

## Abstract (arXiv)

Recent work in Machine Learning and Computer Vision has highlighted the presence of various types of systematic flaws inside ground truth object recognition benchmark datasets. Our basic tenet is that these flaws are rooted in the many-to-many mappings which exist between the visual information encoded in images and the intended semantics of the labels annotating them. The net consequence is that the current annotation process is largely under-specified, thus leaving too much freedom to the subjective judgment of annotators. In this paper, we propose vTelos, an integrated Natural Language Processing, Knowledge Representation, and Computer Vision methodology whose main goal is to make explicit the (otherwise implicit) intended annotation semantics, thus minimizing the number and role of subjective choices. A key element of vTelos is the exploitation of the WordNet lexico-semantic hierarchy as the main means for providing the meaning of natural language labels and, as a consequence, for driving the annotation of images based on the objects and the visual properties they depict. The methodology is validated on images populating a subset of the ImageNet hierarchy.