mlsanityData to 14 Oct 2026

Why KDAC? A general activation function for knowledge discovery

arXiv 2111.13858 ↗First posted 27 Nov 2021cs.LG cs.CL cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep learning oriented named entity recognition (DNER) has gradually become the paradigm of knowledge discovery, which greatly promotes domain intelligence. However, the current activation function of DNER fails to treat gradient vanishing, no negative output or non-differentiable existence, which may impede knowledge exploration caused by the omission and incomplete representation of latent semantics. To break through the dilemma, we present a novel activation function termed KDAC. Detailly, KDAC is an aggregation function with multiple conversion modes. The backbone of the activation region is the interaction between exponent and linearity, and the both ends extend through adaptive linear divergence, which surmounts the obstacle of gradient vanishing and no negative output. Crucially, the non-differentiable points are alerted and eliminated by an approximate smoothing algorithm. KDAC has a series of brilliant properties, including nonlinear, stable near-linear transformation and derivative, as well as dynamic style, etc. We perform experiments based on BERT-BiLSTM-CNN-CRF model on six benchmark datasets containing different domain knowledge, such as Weibo, Clinical, E-commerce, Resume, HAZOP and People's daily. The evaluation results show that KDAC is advanced and effective, and can provide more generalized activation to stimulate the performance of DNER. We hope that KDAC can be exploited as a promising activation function to devote itself to the construction of knowledge.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 24 of its 125 references that are in the map. First posted on arXiv 27 Nov 2021.

Builds on

24 of 125 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  4. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  5. 2017Self-Normalizing Neural Networkscited by 598
  6. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 282
  7. 2019Residual Flows for Invertible Generative Modelingcited by 257
  8. 2013Maxout Networkscited by 187
  9. 2020BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervisioncited by 91
  10. 2020A survey on modern trainable activation functionscited by 89
  11. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  12. 2018Chinese NER Using Lattice LSTMcited by 81
12 more
  1. 2016Noisy Activation Functionscited by 53
  2. 2016Parametric Exponential Linear Unit for Deep Convolutional Neural Networkscited by 32
  3. 2015Deep Learning with S-shaped Rectified Linear Activation Unitscited by 26
  4. 2017xUnit: Learning a Spatial Activation Function for Efficient Image Restorationcited by 10
  5. 2019Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networkscited by 10
  6. 2020Semi-Supervised Disentangled Framework for Transferable Named Entity Recognitioncited by 7
  7. 2021SkillNER: Mining and Mapping Soft Skills from any Textcited by 6
  8. 2018Weighted Sigmoid Gate Unit for an Activation Function of Deep Neural Networkcited by 5
  9. 2020An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognitioncited by 5
  10. 2020ASTRAL: Adversarial Trained LSTM-CNN for Named Entity Recognitioncited by 5
  11. 2020A hybrid deep-learning approach for complex biochemical named entity recognitioncited by 5
  12. 2020Method and Dataset Entity Mining in Scientific Literature: A CNN + Bi-LSTM Model with Self-attentioncited by 3

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 0; 2024: 1.

4 papers.

  1. 2024Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Datacited by 3
  2. 2022A Zipf's Law-based Text Generation Approach for Addressing Imbalance in Entity Extractioncited by 2
  3. 2022A new hazard event classification model via deep learning and multifractalcited by 2
  4. 2022Yes, DLGM! A novel hierarchical model for hazard classificationcited by 0