mlsanityData to 7 Oct 2026

MOLE: MOdular Learning FramEwork via Mutual Information Maximization

arXiv 2308.07772 ↗First posted 15 Aug 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper is to introduce an asynchronous and local learning framework for neural networks, named Modular Learning Framework (MOLE). This framework modularizes neural networks by layers, defines the training objective via mutual information for each module, and sequentially trains each module by mutual information maximization. MOLE makes the training become local optimization with gradient-isolated across modules, and this scheme is more biologically plausible than BP. We run experiments on vector-, grid- and graph-type data. In particular, this framework is capable of solving both graph- and node-level tasks for graph-type data. Therefore, MOLE has been experimentally proven to be universally applicable to different types of data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 44 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

21 of 44 references.

References · 21

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  3. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  4. 2018Deep Graph Infomaxcited by 1,101
  5. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  6. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  7. 2018MINE: Mutual Information Neural Estimationcited by 680
  8. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  9. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  10. 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
  11. 2020Graph Information Bottleneckcited by 168
  12. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
9 more
  1. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  2. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
  3. 2019The HSIC Bottleneck: Deep Learning without Back-Propagationcited by 77
  4. 2018Understanding Convolutional Neural Networks with Information Theory: An Initial Explorationcited by 31
  5. 2018Understanding Autoencoders with Information Theoretic Conceptscited by 29
  6. 2021Measuring Dependence with Matrix-based Entropy Functionalcited by 19
  7. 2023Graph Neural Networks Go Forward-Forwardcited by 8
  8. 2020Modularizing Deep Learning via Pairwise Learning With Kernelscited by 7
  9. 2017Stacked Kernel Networkcited by 1

Built on by

Not cited yet.