# ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition

arXiv: https://arxiv.org/abs/2104.03841 · HTML: https://mlsanity.com/paper/2104.03841/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 60 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)

## Built on by (most cited first, 29 of 29)

- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts](https://mlsanity.com/paper/2210.03885/) (2022)
- [Eliciting and Learning with Soft Labels from Every Annotator](https://mlsanity.com/paper/2207.00810/) (2022)
- [Strong Baselines for Parameter Efficient Few-Shot Fine-tuning](https://mlsanity.com/paper/2304.01917/) (2023)
- [Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions](https://mlsanity.com/paper/2205.13803/) (2022)
- [FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification](https://mlsanity.com/paper/2206.08671/) (2022)
- [Visual Prompting for Generalized Few-shot Segmentation: A Multi-scale Approach](https://mlsanity.com/paper/2404.11732/) (2024)
- [Memory Efficient Meta-Learning with Large Images](https://mlsanity.com/paper/2107.01105/) (2021)
- [On the Efficacy of Differentially Private Few-shot Image Classification](https://mlsanity.com/paper/2302.01190/) (2023)
- [VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairments](https://mlsanity.com/paper/2207.11810/) (2022)
- [Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users](https://mlsanity.com/paper/2503.22610/) (2025)
- [Explaining CLIP's performance disparities on data from blind/low vision users](https://mlsanity.com/paper/2311.17315/) (2023)
- [Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification](https://mlsanity.com/paper/2206.09843/) (2022)
- [Blind Users Accessing Their Training Images in Teachable Object Recognizers](https://mlsanity.com/paper/2208.07968/) (2022)
- [Geoclidean: Few-Shot Generalization in Euclidean Geometry](https://mlsanity.com/paper/2211.16663/) (2022)
- [Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos](https://mlsanity.com/paper/2309.08022/) (2023)
- [Improving ProtoNet for Few-Shot Video Object Recognition: Winner of ORBIT Challenge 2022](https://mlsanity.com/paper/2210.00174/) (2022)
- [BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments](https://mlsanity.com/paper/2407.18243/) (2024)
- [JoLT: Joint Probabilistic Predictions on Tabular Data Using LLMs](https://mlsanity.com/paper/2502.11877/) (2025)
- [Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation](https://mlsanity.com/paper/2306.13554/) (2023)
- [Helping Visually Impaired People Take Better Quality Pictures](https://mlsanity.com/paper/2305.08066/) (2023)
- [A Benchmark Grocery Dataset of Realworld Point Clouds From Single View](https://mlsanity.com/paper/2402.07819/) (2024)
- [Exploring Object Status Recognition for Recipe Progress Tracking in Non-Visual Cooking](https://mlsanity.com/paper/2507.03330/) (2025)
- [How Sensitive are Meta-Learners to Dataset Imbalance?](https://mlsanity.com/paper/2104.05344/) (2021)
- [PRAC3 (Privacy, Reputation, Accountability, Consent, Credit, Compensation): Long Tailed Risks of Voice Actors in AI Data-Economy](https://mlsanity.com/paper/2507.16247/) (2025)
- [AccessShare: Co-designing Data Access and Sharing with Blind People](https://mlsanity.com/paper/2407.19351/) (2024)
- [Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Information](https://mlsanity.com/paper/2512.10102/) (2025)
- [Cripping AI: Reimagining AI Through Lived Disability Experiences](https://mlsanity.com/paper/2605.02080/) (2026)
- [Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed](https://mlsanity.com/paper/2607.24898/) (2026)

## Abstract (arXiv)

Object recognition has made great advances in the last decade, but predominately still relies on many high-quality training examples per object category. In contrast, learning new objects from only a few examples could enable many impactful applications from robotics to user personalization. Most few-shot learning research, however, has been driven by benchmark datasets that lack the high variation that these applications will face when deployed in the real-world. To close this gap, we present the ORBIT dataset and benchmark, grounded in the real-world application of teachable object recognizers for people who are blind/low-vision. The dataset contains 3,822 videos of 486 objects recorded by people who are blind/low-vision on their mobile phones. The benchmark reflects a realistic, highly challenging recognition problem, providing a rich playground to drive research in robustness to few-shot, high-variation conditions. We set the benchmark's first state-of-the-art and show there is massive scope for further innovation, holding the potential to impact a broad range of real-world vision applications including tools for the blind/low-vision community. We release the dataset at https://doi.org/10.25383/city.14294597 and benchmark code at https://github.com/microsoft/ORBIT-Dataset.