# FOCAL: A Cost-Aware Video Dataset for Active Learning

arXiv: https://arxiv.org/abs/2311.10591 · HTML: https://mlsanity.com/paper/2311.10591/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 19 of 19)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Scalable Active Learning for Object Detection](https://mlsanity.com/paper/2004.04699/) (2020)
- [Active Learning with Partial Feedback](https://mlsanity.com/paper/1802.07427/) (2018)
- [A Machine Learning Benchmark for Facies Classification](https://mlsanity.com/paper/1901.07659/) (2019)
- [Temporal Coherence for Active Learning in Videos](https://mlsanity.com/paper/1908.11757/) (2019)
- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [Patient Aware Active Learning for Fine-Grained OCT Classification](https://mlsanity.com/paper/2206.11485/) (2022)
- [Clinically Labeled Contrastive Learning for OCT Biomarker Classification](https://mlsanity.com/paper/2305.15154/) (2023)
- [Forgetful Active Learning with Switch Events: Efficient Sampling for Out-of-Distribution Data](https://mlsanity.com/paper/2301.05106/) (2023)
- [DECAL: DEployable Clinical Active Learning](https://mlsanity.com/paper/2206.10120/) (2022)
- [Man-recon: manifold learning for reconstruction with deep autoencoder for smart seismic interpretation](https://mlsanity.com/paper/2212.07568/) (2022)
- [Exploiting the Distortion-Semantic Interaction in Fisheye Data](https://mlsanity.com/paper/2305.00079/) (2023)

## Built on by (most cited first, 9 of 9)

- [Hierarchical and Multimodal Data for Daily Activity Understanding](https://mlsanity.com/paper/2504.17696/) (2025)
- [A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior](https://mlsanity.com/paper/2505.08585/) (2025)
- [Subject Invariant Contrastive Learning for Human Activity Recognition](https://mlsanity.com/paper/2507.03250/) (2025)
- [Targeting Negative Flips in Active Learning using Validation Sets](https://mlsanity.com/paper/2411.10896/) (2024)
- [Countering Multi-modal Representation Collapse through Rank-targeted Fusion](https://mlsanity.com/paper/2511.06450/) (2025)
- [BALD-SAM: Disagreement-based Active Prompting in Interactive Segmentation](https://mlsanity.com/paper/2603.10828/) (2026)
- [Explaining Representation Learning with Perceptual Components](https://mlsanity.com/paper/2406.06930/) (2024)
- [ChildGaze: A Benchmark Dataset for Collaborative Behavior Understanding in Children](https://mlsanity.com/paper/2609.06353/) (2026)
- [RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition](https://mlsanity.com/paper/2609.06843/) (2026)

## Abstract (arXiv)

In this paper, we introduce the FOCAL (Ford-OLIVES Collaboration on Active Learning) dataset which enables the study of the impact of annotation-cost within a video active learning setting. Annotation-cost refers to the time it takes an annotator to label and quality-assure a given video sequence. A practical motivation for active learning research is to minimize annotation-cost by selectively labeling informative samples that will maximize performance within a given budget constraint. However, previous work in video active learning lacks real-time annotation labels for accurately assessing cost minimization and instead operates under the assumption that annotation-cost scales linearly with the amount of data to annotate. This assumption does not take into account a variety of real-world confounding factors that contribute to a nonlinear cost such as the effect of an assistive labeling tool and the variety of interactions within a scene such as occluded objects, weather, and motion of objects. FOCAL addresses this discrepancy by providing real annotation-cost labels for 126 video sequences across 69 unique city scenes with a variety of weather, lighting, and seasonal conditions. We also introduce a set of conformal active learning algorithms that take advantage of the sequential structure of video data in order to achieve a better trade-off between annotation-cost and performance while also reducing floating point operations (FLOPS) overhead by at least 77.67%. We show how these approaches better reflect how annotations on videos are done in practice through a sequence selection framework. We further demonstrate the advantage of these approaches by introducing two performance-cost metrics and show that the best conformal active learning method is cheaper than the best traditional active learning method by 113 hours.