mlsanityData to 14 Oct 2026

Knowledge accumulating: The general pattern of learning

arXiv 2108.03988 ↗First posted 9 Aug 2021cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence has been developed for decades with the achievement of great progress. Recently, deep learning shows its ability to solve many real world problems, e.g. image classification and detection, natural language processing, playing GO. Theoretically speaking, an artificial neural network can fit any function and reinforcement learning can learn from any delayed reward. But in solving real world tasks, we still need to spend a lot of effort to adjust algorithms to fit task unique features. This paper proposes that the reason of this phenomenon is the sparse feedback feature of the nature, and a single algorithm, no matter how we improve it, can only solve dense feedback tasks or specific sparse feedback tasks. This paper first analyses how sparse feedback affects algorithm perfomance, and then proposes a pattern that explains how to accumulate knowledge to solve sparse feedback problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 56 references that are in the map. First posted on arXiv 9 Aug 2021.

Builds on

30 of 56 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  8. 2017Mask R-CNNcited by 7,277
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  12. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
18 more
  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  2. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  3. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  4. 2016Matching Networks for One Shot Learningcited by 2,255
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  8. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  9. 2017Focal Loss for Dense Object Detectioncited by 752
  10. 2017Dynamic Routing Between Capsulescited by 689
  11. 2014Neural Turing Machinescited by 644
  12. 2017Wasserstein GANcited by 639
  13. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  14. 2020What is being transferred in transfer learning?cited by 368
  15. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 137
  16. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92
  17. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  18. 2021Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPscited by 5

Built on by

Not cited yet.