# On the effectiveness of task granularity for transfer learning

arXiv: https://arxiv.org/abs/1804.09235 · HTML: https://mlsanity.com/paper/1804.09235/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2018.

## Built on by (most cited first, 30 of 35)

- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input](https://mlsanity.com/paper/2408.15542/) (2024)
- [EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos](https://mlsanity.com/paper/2507.12440/) (2025)
- [PEEKABOO: Interactive Video Generation via Masked-Diffusion](https://mlsanity.com/paper/2312.07509/) (2023)
- [Video Transformers: A Survey](https://mlsanity.com/paper/2201.05991/) (2022)
- [LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding](https://mlsanity.com/paper/2501.08282/) (2025)
- [AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition](https://mlsanity.com/paper/2102.05775/) (2021)
- [Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback](https://mlsanity.com/paper/2412.02617/) (2024)
- [VideoLT: Large-scale Long-tailed Video Recognition](https://mlsanity.com/paper/2105.02668/) (2021)
- [Large-scale Robustness Analysis of Video Action Recognition Models](https://mlsanity.com/paper/2207.01398/) (2022)
- [Hierarchical Masked 3D Diffusion Model for Video Outpainting](https://mlsanity.com/paper/2309.02119/) (2023)
- [RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation](https://mlsanity.com/paper/2509.15212/) (2025)
- [Learning Representational Invariances for Data-Efficient Action Recognition](https://mlsanity.com/paper/2103.16565/) (2021)
- [Group Contextualization for Video Recognition](https://mlsanity.com/paper/2203.09694/) (2022)
- [Learning Correlation Structures for Vision Transformers](https://mlsanity.com/paper/2404.03924/) (2024)
- [Attention in Attention: Modeling Context Correlation for Efficient Video Classification](https://mlsanity.com/paper/2204.09303/) (2022)
- [Taylor Videos for Action Recognition](https://mlsanity.com/paper/2402.03019/) (2024)
- [Detecting Object States vs Detecting Objects: A New Dataset and a Quantitative Experimental Study](https://mlsanity.com/paper/2112.08281/) (2021)
- [Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders](https://mlsanity.com/paper/2510.26027/) (2025)
- [A Comprehensive Review of the Video-to-Text Problem](https://mlsanity.com/paper/2103.14785/) (2021)
- [Video Is Graph: Structured Graph Module for Video Action Recognition](https://mlsanity.com/paper/2110.05904/) (2021)
- [Zero-shot Compositional Action Recognition with Neural Logic Constraints](https://mlsanity.com/paper/2508.02320/) (2025)
- [The Multi-Modal Video Reasoning and Analyzing Competition](https://mlsanity.com/paper/2108.08344/) (2021)
- [Selective Volume Mixup for Video Action Recognition](https://mlsanity.com/paper/2309.09534/) (2023)
- [UFVideo: Towards Unified Fine-Grained Video Cooperative Understanding with Large Language Models](https://mlsanity.com/paper/2512.11336/) (2025)
- [Ensemble Modeling for Multimodal Visual Action Recognition](https://mlsanity.com/paper/2308.05430/) (2023)
- [Egocentric RGB+Depth Action Recognition in Industry-Like Settings](https://mlsanity.com/paper/2309.13962/) (2023)
- [Exploring High-Order Self-Similarity for Video Understanding](https://mlsanity.com/paper/2604.20760/) (2026)
- [Gamified crowd-sourcing of high-quality data for visual fine-tuning](https://mlsanity.com/paper/2410.04038/) (2024)
- [An Efficient 3D Convolutional Neural Network with Channel-wise, Spatial-grouped, and Temporal Convolutions](https://mlsanity.com/paper/2503.00796/) (2025)

## Abstract (arXiv)

We describe a DNN for video classification and captioning, trained end-to-end, with shared features, to solve tasks at different levels of granularity, exploring the link between granularity in a source task and the quality of learned features for transfer learning. For solving the new task domain in transfer learning, we freeze the trained encoder and fine-tune a neural net on the target domain. We train on the Something-Something dataset with over 220, 000 videos, and multiple levels of target granularity, including 50 action groups, 174 fine-grained action categories and captions. Classification and captioning with Something-Something are challenging because of the subtle differences between actions, applied to thousands of different object classes, and the diversity of captions penned by crowd actors. Our model performs better than existing classification baselines for SomethingSomething, with impressive fine-grained results. And it yields a strong baseline on the new Something-Something captioning task. Experiments reveal that training with more fine-grained tasks tends to produce better features for transfer learning.