# Motif Mining: Finding and Summarizing Remixed Image Content

arXiv: https://arxiv.org/abs/2203.08327 · HTML: https://mlsanity.com/paper/2203.08327/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [Deep Spectral Clustering using Dual Autoencoder Network](https://mlsanity.com/paper/1904.13113/) (2019)
- [On the Origins of Memes by Means of Fringe Web Communities](https://mlsanity.com/paper/1805.12512/) (2018)
- [Online Invariance Selection for Local Feature Descriptors](https://mlsanity.com/paper/2007.08988/) (2020)
- [Visual Psychophysics for Making Face Recognition Algorithms More Explainable](https://mlsanity.com/paper/1803.07140/) (2018)
- [PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition](https://mlsanity.com/paper/1611.06448/) (2016)
- [Image Provenance Analysis at Scale](https://mlsanity.com/paper/1801.06510/) (2018)
- [MemeSequencer: Sparse Matching for Embedding Image Macros](https://mlsanity.com/paper/1802.04936/) (2018)
- [Deep Image Clustering with Category-Style Representation](https://mlsanity.com/paper/2007.10004/) (2020)
- [Automatic Discovery of Political Meme Genres with Diverse Appearances](https://mlsanity.com/paper/2001.06122/) (2020)
- [Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation](https://mlsanity.com/paper/1709.01220/) (2017)

## Built on by (most cited first, 7 of 7)

- [C-CLIP: Contrastive Image-Text Encoders to Close the Descriptive-Commentative Gap](https://mlsanity.com/paper/2309.03921/) (2023)
- [Fear and Loathing on the Frontline: Decoding the Language of Othering by Russia-Ukraine War Bloggers](https://mlsanity.com/paper/2409.13064/) (2024)
- [Needle In A Haystack, Fast: Benchmarking Image Perceptual Similarity Metrics At Scale](https://mlsanity.com/paper/2206.00282/) (2022)
- [A Volumetric Saliency Guided Image Summarization for RGB-D Indoor Scene Classification](https://mlsanity.com/paper/2401.16227/) (2024)
- [N-Modal Contrastive Losses with Applications to Social Media Data in Trimodal Space](https://mlsanity.com/paper/2403.12747/) (2024)
- [Clustering Internet Memes Through Template Matching and Multi-Dimensional Similarity](https://mlsanity.com/paper/2505.00056/) (2025)
- [Modeling Information Narrative Detection and Evolution on Telegram during the Russia-Ukraine War](https://mlsanity.com/paper/2409.07684/) (2024)

## Abstract (arXiv)

On the internet, images are no longer static; they have become dynamic content. Thanks to the availability of smartphones with cameras and easy-to-use editing software, images can be remixed (i.e., redacted, edited, and recombined with other content) on-the-fly and with a world-wide audience that can repeat the process. From digital art to memes, the evolution of images through time is now an important topic of study for digital humanists, social scientists, and media forensics specialists. However, because typical data sets in computer vision are composed of static content, the development of automated algorithms to analyze remixed content has been limited. In this paper, we introduce the idea of Motif Mining - the process of finding and summarizing remixed image content in large collections of unlabeled and unsorted data. In this paper, this idea is formalized and a reference implementation is introduced. Experiments are conducted on three meme-style data sets, including a newly collected set associated with the information war in the Russo-Ukrainian conflict. The proposed motif mining approach is able to identify related remixed content that, when compared to similar approaches, more closely aligns with the preferences and expectations of human observers.