mlsanityData to 7 Oct 2026

Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning

arXiv 2406.16949 ↗First posted 19 Jun 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advancements in artificial intelligence (AI) have positioned deep learning (DL) as a pivotal technology in fields like computer vision, data mining, and natural language processing. A critical factor in DL performance is the selection of neural network architecture. Traditional predefined architectures often fail to adapt to different data distributions, making it challenging to achieve optimal performance. Neural architecture search (NAS) offers a solution by automatically designing architectures tailored to specific datasets. However, the effectiveness of NAS diminishes on long-tailed datasets, where a few classes have abundant samples, and many have few, leading to biased models.In this paper, we explore to improve the searching and training performance of NAS on long-tailed datasets. Specifically, we first discuss the related works about NAS and the deep learning method for long-tailed datasets.Then, we focus on an existing work, called SSF-NAS, which integrates the self-supervised learning and fair differentiable NAS to making NAS achieve better performance on long-tailed datasets.An detailed description about the fundamental techniques for SSF-NAS is provided in this paper, including DARTS, FairDARTS, and Barlow Twins. Finally, we conducted a series of experiments on the CIFAR10-LT dataset for performance evaluation, where the results are align with our expectation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

28 of 35 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  7. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  8. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  9. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  10. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  11. 2021Deep Long-Tailed Learning: A Surveycited by 416
  12. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
16 more
  1. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  2. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  3. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  4. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  5. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  6. 2018Gradient Harmonized Single-stage Detectorcited by 114
  7. 2017Evolving Deep Neural Networkscited by 93
  8. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  9. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  10. 2018Automatically designing CNN architectures using genetic algorithm for image classificationcited by 48
  11. 2020From Federated Learning to Federated Neural Architecture Search: A Surveycited by 42
  12. 2018BlockQNN: Efficient Block-wise Neural Network Architecture Generationcited by 31
  13. 2020Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Searchcited by 30
  14. 2021Self-Supervised Neural Architecture Search for Imbalanced Datasetscited by 4
  15. 2018Finding Better Topologies for Deep Convolutional Neural Networks by Evolutioncited by 2
  16. 2022IMB-NAS: Neural Architecture Search for Imbalanced Datasetscited by 2

Built on by

Not cited yet.