# Optimize TSK Fuzzy Systems for Regression Problems: Mini-Batch Gradient Descent with Regularization, DropRule and AdaBound (MBGD-RDA)

arXiv: https://arxiv.org/abs/1903.10951 · HTML: https://mlsanity.com/paper/1903.10951/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 9 of 9)

- [Curse of Dimensionality for TSK Fuzzy Neural Networks: Explanation and Solutions](https://mlsanity.com/paper/2102.04271/) (2021)
- [An Adaptive Neuro-Fuzzy System with Integrated Feature Selection and Rule Extraction for High-Dimensional Classification Problems](https://mlsanity.com/paper/2201.03187/) (2022)
- [PyTSK: A Python Toolbox for TSK Fuzzy Systems](https://mlsanity.com/paper/2206.03310/) (2022)
- [BoxSeg: Quality-Aware and Peer-Assisted Learning for Box-supervised Instance Segmentation](https://mlsanity.com/paper/2504.05137/) (2025)
- [Fuzzy Knowledge Distillation from High-Order TSK to Low-Order TSK](https://mlsanity.com/paper/2302.08038/) (2023)
- [Gradient-based Fuzzy System Optimisation via Automatic Differentiation -- FuzzyR as a Use Case](https://mlsanity.com/paper/2403.12308/) (2024)
- [A Self-Constructing Multi-Expert Fuzzy System for High-dimensional Data Classification](https://mlsanity.com/paper/2410.13390/) (2024)
- [An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models](https://mlsanity.com/paper/2511.08077/) (2025)
- [Alternating Bi-Objective Optimization for Explainable Neuro-Fuzzy Systems](https://mlsanity.com/paper/2602.19253/) (2026)

## Abstract (arXiv)

Takagi-Sugeno-Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that ensures their generalization performance, and also enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., mini-batch gradient descent, regularization, and AdaBound, to TSK fuzzy systems, and also propose three novel techniques (DropRule, DropMF, and DropMembership) specifically for training TSK fuzzy systems. Our final algorithm, mini-batch gradient descent with regularization, DropRule and AdaBound (MBGD-RDA), can achieve fast convergence in training TSK fuzzy systems, and also superior generalization performance in testing. It can be used for training TSK fuzzy systems on datasets of any size; however, it is particularly useful for big datasets, on which currently no other efficient training algorithms exist.