# Automatic Image Content Extraction: Operationalizing Machine Learning in Humanistic Photographic Studies of Large Visual Archives

arXiv: https://arxiv.org/abs/2204.02149 · HTML: https://mlsanity.com/paper/2204.02149/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 82 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation](https://mlsanity.com/paper/1707.09423/) (2017)
- [Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)](https://mlsanity.com/paper/2001.00139/) (2020)
- [Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning](https://mlsanity.com/paper/1802.05800/) (2018)
- [StreetStyle: Exploring world-wide clothing styles from millions of photos](https://mlsanity.com/paper/1706.01869/) (2017)
- [A CNN-RNN Architecture for Multi-Label Weather Recognition](https://mlsanity.com/paper/1904.10709/) (2019)
- [The Visual Social Distancing Problem](https://mlsanity.com/paper/2005.04813/) (2020)
- [Machine Learning Based Analysis of Finnish World War II Photographers](https://mlsanity.com/paper/1904.09811/) (2019)
- [Automatic Social Distance Estimation From Images: Performance Evaluation, Test Benchmark, and Algorithm](https://mlsanity.com/paper/2103.06759/) (2021)
- [Transferable Interactiveness Knowledge for Human-Object Interaction Detection](https://mlsanity.com/paper/2101.10292/) (2021)
- [Automatic Main Character Recognition for Photographic Studies](https://mlsanity.com/paper/2106.09064/) (2021)

## Built on by (most cited first, 1 of 1)

- [For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives](https://mlsanity.com/paper/2407.03268/) (2024)

## Abstract (arXiv)

Applying machine learning tools to digitized image archives has a potential to revolutionize quantitative research of visual studies in humanities and social sciences. The ability to process a hundredfold greater number of photos than has been traditionally possible and to analyze them with an extensive set of variables will contribute to deeper insight into the material. Overall, these changes will help to shift the workflow from simple manual tasks to more demanding stages. In this paper, we introduce Automatic Image Content Extraction (AICE) framework for machine learning-based search and analysis of large image archives. We developed the framework in a multidisciplinary research project as framework for future photographic studies by reformulating and expanding the traditional visual content analysis methodologies to be compatible with the current and emerging state-of-the-art machine learning tools and to cover the novel machine learning opportunities for automatic content analysis. The proposed framework can be applied in several domains in humanities and social sciences, and it can be adjusted and scaled into various research settings. We also provide information on the current state of different machine learning techniques and show that there are already various publicly available methods that are suitable to a wide-scale of visual content analysis tasks.