mlsanityData to 14 Oct 2026

Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast

arXiv 2106.03796 ↗First posted 7 Jun 2021cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and storages of edge devices are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

19 of 26 references.

References · 19

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2019Towards Federated Learning at Scale: System Designcited by 832
  5. 2019Gradient based sample selection for online continual learningcited by 571
  6. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  7. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  8. 2020Coresets via Bilevel Optimization for Continual Learning and Streamingcited by 192
  9. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  10. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  11. 2020Self-supervised learning through the eyes of a childcited by 61
  12. 2019Hardware/Software Co-Exploration of Neural Architecturescited by 50
7 more
  1. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 38
  2. 2019Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Searchcited by 38
  3. 2019OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learningcited by 28
  4. 2020Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruningcited by 24
  5. 2020Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devicescited by 19
  6. 2020Temporally Coherent Embeddings for Self-Supervised Video Representation Learningcited by 17
  7. 2018A Survey of Deep Learning Techniques for Mobile Robot Applicationscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 6; 2023: 6; 2024: 7.

21 papers.

Sort:

  1. 2022Federated Contrastive Learning for Volumetric Medical Image Segmentationcited by 33
  2. 2023AutoReP: Automatic ReLU Replacement for Fast Private Network Inferencecited by 29
  3. 2023LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inferencecited by 27
  4. 2022Federated Contrastive Learning for Dermatological Disease Diagnosis via On-device Learningcited by 21
  5. 2024Unlocking Memorization in Large Language Models with Dynamic Soft Promptingcited by 21
  6. 2023Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesiscited by 20
  7. 2022Distributed Contrastive Learning for Medical Image Segmentationcited by 16
  8. 2021Decentralized Unsupervised Learning of Visual Representationscited by 15
  9. 2024SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezingcited by 12
  10. 2021Exploration of Quantum Neural Architecture by Mixing Quantum Neuron Designscited by 11
  11. 2024Training Machine Learning models at the Edge: A Surveycited by 11
  12. 2023BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphscited by 7
9 more
  1. 2022Self-supervised On-device Federated Learning from Unlabeled Streamscited by 6
  2. 2023QuMoS: A Framework for Preserving Security of Quantum Machine Learning Modelcited by 6
  3. 2022Federated Self-Supervised Contrastive Learning and Masked Autoencoder for Dermatological Disease Diagnosiscited by 5
  4. 2024All-in-One Tuning and Structural Pruning for Domain-Specific LLMscited by 4
  5. 2023Edge-InversionNet: Enabling Efficient Inference of InversionNet on Edge Devicescited by 2
  6. 2024EdgeOL: Efficient in-situ Online Learning on Edge Devicescited by 2
  7. 2024A Self-guided Multimodal Approach to Enhancing Graph Representation Learning for Alzheimer's Diseasescited by 1
  8. 2022Enabling Weakly-Supervised Temporal Action Localization from On-Device Learning of the Video Streamcited by 0
  9. 2024Enabling On-Device Learning via Experience Replay with Efficient Dataset Condensationcited by 0