mlsanityData to 7 Oct 2026

Matryoshka Representation Learning

arXiv 2205.13147 ↗First posted 26 May 2022cs.LG cs.CVcited by 268

Not on the map. Why?

Abstract (arXiv)

Learned representations are a central component in modern ML systems, serving a multitude of downstream tasks. When training such representations, it is often the case that computational and statistical constraints for each downstream task are unknown. In this context rigid, fixed capacity representations can be either over or under-accommodating to the task at hand. This leads us to ask: can we design a flexible representation that can adapt to multiple downstream tasks with varying computational resources? Our main contribution is Matryoshka Representation Learning (MRL) which encodes information at different granularities and allows a single embedding to adapt to the computational constraints of downstream tasks. MRL minimally modifies existing representation learning pipelines and imposes no additional cost during inference and deployment. MRL learns coarse-to-fine representations that are at least as accurate and rich as independently trained low-dimensional representations. The flexibility within the learned Matryoshka Representations offer: (a) up to 14x smaller embedding size for ImageNet-1K classification at the same level of accuracy; (b) up to 14x real-world speed-ups for large-scale retrieval on ImageNet-1K and 4K; and (c) up to 2% accuracy improvements for long-tail few-shot classification, all while being as robust as the original representations. Finally, we show that MRL extends seamlessly to web-scale datasets (ImageNet, JFT) across various modalities -- vision (ViT, ResNet), vision + language (ALIGN) and language (BERT). MRL code and pretrained models are open-sourced at https://github.com/RAIVNLab/MRL.

Key facts. Cited by 268 papers in the map (data through 7 Oct 2026). Builds on 54 of its 116 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

54 of 116 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
38 more
  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  9. 2019Natural Adversarial Examplescited by 1,401
  10. 2014How transferable are features in deep neural networks?cited by 1,216
  11. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  13. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  14. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  15. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  16. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  17. 2021Robust fine-tuning of zero-shot modelscited by 785
  18. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  19. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  20. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  21. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  22. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  23. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  24. 2018Slimmable Neural Networkscited by 301
  25. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  26. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  27. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  28. 2020Characterising Bias in Compressed Modelscited by 162
  29. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  30. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  31. 2014Learning Ordered Representations with Nested Dropoutcited by 81
  32. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  33. 2018On the Intrinsic Dimensionality of Image Representationscited by 57
  34. 2020PECOS: Prediction for Enormous and Correlated Output Spacescited by 50
  35. 2022Task Adaptive Parameter Sharing for Multi-Task Learningcited by 42
  36. 2019FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Networkcited by 38
  37. 2021Extreme Multi-label Learning for Semantic Matching in Product Searchcited by 28
  38. 2020HERS: Homomorphically Encrypted Representation Searchcited by 26

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 35; 2025: 91; 2026: 135.

268 papers.

Sort:

By area:

  1. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  2. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  3. 2023Hyperbolic Image-Text Representationscited by 132
  4. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  5. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  6. 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 95
  7. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 93
  8. 2024Matryoshka Multimodal Modelscited by 82
  9. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  10. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 79
  11. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  12. 2023Matryoshka Diffusion Modelscited by 70
38 more
  1. 2024Vlogger: Make Your Dream A Vlogcited by 70
  2. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  3. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  4. 2024ElasticTok: Adaptive Tokenization for Image and Videocited by 44
  5. 2024Flextron: Many-in-One Flexible Large Language Modelcited by 42
  6. 2024Adaptive Length Image Tokenization via Recurrent Allocationcited by 40
  7. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 36
  8. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  9. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  10. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  11. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projectioncited by 29
  12. 2024Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunicationscited by 26
  13. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 25
  14. 2025"Principal Components" Enable A New Language of Imagescited by 25
  15. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  16. 2025Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learningcited by 21
  17. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
  18. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmarkcited by 19
  19. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  20. 2025Post-training Large Language Models for Diverse High-Quality Responsescited by 19
  21. 2026jina-embeddings-v5-text: Task-Targeted Embedding Distillationcited by 19
  22. 2024Non-negative Contrastive Learningcited by 18
  23. 2024LOLA: LLM-Assisted Online Learning Algorithm for Content Experimentscited by 18
  24. 2025CAT: Content-Adaptive Image Tokenizationcited by 18
  25. 2022Neural Eigenfunctions Are Structured Representation Learnerscited by 16
  26. 2024KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledgecited by 16
  27. 2025Hypergraph Vision Transformers: Images are More than Nodes, More than Edgescited by 16
  28. 2024Cascade-Aware Training of Language Modelscited by 15
  29. 2024Conan-embedding: General Text Embedding with More and Better Negative Samplescited by 15
  30. 2024Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systemscited by 15
  31. 2025Table Foundation Models: on knowledge pre-training for tabular learningcited by 15
  32. 2026OAT: Ordered Action Tokenizationcited by 14
  33. 2024Operator SVD with Neural Networks via Nested Low-Rank Approximationcited by 12
  34. 2025R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Expertscited by 12
  35. 2025Efficient Code Embeddings from Code Generation Modelscited by 12
  36. 2024Crafting Interpretable Embeddings by Asking LLMs Questionscited by 11
  37. 2024CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agentscited by 11
  38. 2025Matryoshka Quantizationcited by 11

50 most cited of 268 All 268 (JSON)