# Representation Learning on Large and Small Data

arXiv: https://arxiv.org/abs/1707.09873 · HTML: https://mlsanity.com/paper/1707.09873/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2017.

## Built on by (most cited first, 2 of 2)

- [Knowledge-Guided Data-Centric AI in Healthcare: Progress, Shortcomings, and Future Directions](https://mlsanity.com/paper/2212.13591/) (2022)
- [Active Learning on Neural Networks through Interactive Generation of Digit Patterns and Visual Representation](https://mlsanity.com/paper/2310.01580/) (2023)

## Abstract (arXiv)

Deep learning owes its success to three key factors: scale of data, enhanced models to learn representations from data, and scale of computation. This book chapter presented the importance of the data-driven approach to learn good representations from both big data and small data. In terms of big data, it has been widely accepted in the research community that the more data the better for both representation and classification improvement. The question is then how to learn representations from big data, and how to perform representation learning when data is scarce. We addressed the first question by presenting CNN model enhancements in the aspects of representation, optimization, and generalization. To address the small data challenge, we showed transfer representation learning to be effective. Transfer representation learning transfers the learned representation from a source domain where abundant training data is available to a target domain where training data is scarce. Transfer representation learning gave the OM and melanoma diagnosis modules of our XPRIZE Tricorder device (which finished 2nd out of 310 competing teams) a significant boost in diagnosis accuracy.