mlsanityData to 14 Oct 2026

LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machines

arXiv 2604.12223 ↗First posted 14 Apr 2026cs.CL cs.AI cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Pretrained language models (PLMs) like BERT provide strong semantic representations but are costly and opaque, while symbolic models such as the Tsetlin Machine (TM) offer transparency but lack semantic generalization. We propose a semantic bootstrapping framework that transfers LLM knowledge into symbolic form, combining interpretability with semantic capacity. Given a class label, an LLM generates sub-intents that guide synthetic data creation through a three-stage curriculum (seed, core, enriched), expanding semantic diversity. A Non-Negated TM (NTM) learns from these examples to extract high-confidence literals as interpretable semantic cues. Injecting these cues into real data enables a TM to align clause logic with LLM-inferred semantics. Our method requires no embeddings or runtime LLM calls, yet equips symbolic models with pretrained semantic priors. Across multiple text classification tasks, it improves interpretability and accuracy over vanilla TM, achieving performance comparable to BERT while remaining fully symbolic and efficient.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 26 of its 45 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

26 of 45 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2019Attention is not Explanationcited by 929
  6. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  7. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  8. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 299
  9. 2019Fine-tune BERT for Extractive Summarizationcited by 122
  10. 2018The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logiccited by 72
  11. 2025The Price of Format: Diversity Collapse in LLMscited by 43
  12. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 39
14 more
  1. 2019Fine-grained Sentiment Classification using BERTcited by 38
  2. 2023Large Language Models Are Zero-Shot Text Classifierscited by 35
  3. 2018Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applicationscited by 24
  4. 2021Explainable Tsetlin Machine framework for fake news detection with credibility score assessmentcited by 23
  5. 2020Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretabilitycited by 16
  6. 2023Tsetlin Machine Embedding: Representing Words Using Logical Expressionscited by 15
  7. 2023Building Concise Logical Patterns by Constraining Tsetlin Machine Clause Sizecited by 14
  8. 2025The Confidence Paradox: Can LLM Know When It's Wrongcited by 11
  9. 2021Enhancing Interpretable Clauses Semantically using Pretrained Word Representationcited by 10
  10. 2022Transparency Helps Reveal When Language Models Learn Meaningcited by 8
  11. 2024Exploring Empty Spaces: Human-in-the-Loop Data Augmentationcited by 6
  12. 2019A Tsetlin Machine with Multigranular Clausescited by 5
  13. 2024Exploring State Space and Reasoning by Elimination in Tsetlin Machinescited by 4
  14. 2025Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decompositioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
  2. 2022Augmenting Interpretable Models with LLMs during Training
  3. 2023Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Models
  4. 2024Concept Bottleneck Large Language Models
  5. 2024Rethinking Interpretability in the Era of Large Language Models
  6. 2024Verbalized Machine Learning: Revisiting Machine Learning with Language Models
  7. 2024Large Language Models are Interpretable Learners
  8. 2026A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification