# BI-LAVA: Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analysis

arXiv: https://arxiv.org/abs/2308.08003 · HTML: https://mlsanity.com/paper/2308.08003/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 85 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Active Learning by Acquiring Contrastive Examples](https://mlsanity.com/paper/2109.03764/) (2021)
- [A Survey of Visual Analytics Techniques for Machine Learning](https://mlsanity.com/paper/2008.09632/) (2020)
- [t-viSNE: Interactive Assessment and Interpretation of t-SNE Projections](https://mlsanity.com/paper/2002.06910/) (2020)
- [OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples](https://mlsanity.com/paper/2002.03103/) (2020)
- [VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection](https://mlsanity.com/paper/2009.12975/) (2020)
- [Visualizing Data using GTSNE](https://mlsanity.com/paper/2108.01301/) (2021)
- [A survey of image labelling for computer vision applications](https://mlsanity.com/paper/2104.08885/) (2021)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [How to Manipulate CNNs to Make Them Lie: the GradCAM Case](https://mlsanity.com/paper/1907.10901/) (2019)
- [DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps](https://mlsanity.com/paper/2205.06935/) (2022)
- [VIANA: Visual Interactive Annotation of Argumentation](https://mlsanity.com/paper/1907.12413/) (2019)
- [VASSL: A Visual Analytics Toolkit for Social Spambot Labeling](https://mlsanity.com/paper/1907.13319/) (2019)
- [A Generic Framework and Library for Exploration of Small Multiples through Interactive Piling](https://mlsanity.com/paper/2005.00595/) (2020)

## Abstract (arXiv)

In the biomedical domain, taxonomies organize the acquisition modalities of scientific images in hierarchical structures. Such taxonomies leverage large sets of correct image labels and provide essential information about the importance of a scientific publication, which could then be used in biocuration tasks. However, the hierarchical nature of the labels, the overhead of processing images, the absence or incompleteness of labeled data, and the expertise required to label this type of data impede the creation of useful datasets for biocuration. From a multi-year collaboration with biocurators and text-mining researchers, we derive an iterative visual analytics and active learning strategy to address these challenges. We implement this strategy in a system called BI-LAVA Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analysis. BI-LAVA leverages a small set of image labels, a hierarchical set of image classifiers, and active learning to help model builders deal with incomplete ground-truth labels, target a hierarchical taxonomy of image modalities, and classify a large pool of unlabeled images. BI-LAVA's front end uses custom encodings to represent data distributions, taxonomies, image projections, and neighborhoods of image thumbnails, which help model builders explore an unfamiliar image dataset and taxonomy and correct and generate labels. An evaluation with machine learning practitioners shows that our mixed human-machine approach successfully supports domain experts in understanding the characteristics of classes within the taxonomy, as well as validating and improving data quality in labeled and unlabeled collections.