# A Relational Tsetlin Machine with Applications to Natural Language Understanding

arXiv: https://arxiv.org/abs/2102.10952 · HTML: https://mlsanity.com/paper/2102.10952/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 22 Feb 2021.

## Builds on (most cited first, 17 of 17)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic](https://mlsanity.com/paper/1804.01508/) (2018)
- [Turning 30: New Ideas in Inductive Logic Programming](https://mlsanity.com/paper/2002.11002/) (2020)
- [The Convolutional Tsetlin Machine](https://mlsanity.com/paper/1905.09688/) (2019)
- [Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications](https://mlsanity.com/paper/1809.04547/) (2018)
- [Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability](https://mlsanity.com/paper/2005.05131/) (2020)
- [Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin Machine Text Classifier](https://mlsanity.com/paper/2011.08755/) (2020)
- [Low-Power Audio Keyword Spotting using Tsetlin Machines](https://mlsanity.com/paper/2101.11336/) (2021)
- [Closed-Form Expressions for Global and Local Interpretation of Tsetlin Machines with Applications to Explaining High-Dimensional Data](https://mlsanity.com/paper/2007.13885/) (2020)
- [Probabilistic Inductive Logic Programming Based on Answer Set Programming](https://mlsanity.com/paper/1405.0720/) (2014)
- [Machine Reasoning Explainability](https://mlsanity.com/paper/2009.00418/) (2020)
- [The Weighted Tsetlin Machine: Compressed Representations with Weighted Clauses](https://mlsanity.com/paper/1911.12607/) (2019)

## Built on by (most cited first, 8 of 8)

- [Building Concise Logical Patterns by Constraining Tsetlin Machine Clause Size](https://mlsanity.com/paper/2301.08190/) (2023)
- [TMComposites: Plug-and-Play Collaboration Between Specialized Tsetlin Machines](https://mlsanity.com/paper/2309.04801/) (2023)
- [An FPGA Architecture for Online Learning using the Tsetlin Machine](https://mlsanity.com/paper/2306.01027/) (2023)
- [TPFL: Tsetlin-Personalized Federated Learning with Confidence-Based Clustering](https://mlsanity.com/paper/2409.10392/) (2024)
- [Uncertainty Quantification in the Tsetlin Machine](https://mlsanity.com/paper/2507.04175/) (2025)
- [Learning Minimalistic Tsetlin Machine Clauses with Markov Boundary-Guided Pruning](https://mlsanity.com/paper/2309.06315/) (2023)
- [Efficient Data Fusion using the Tsetlin Machine](https://mlsanity.com/paper/2310.17207/) (2023)
- [A Methodology for Transparent Logic-Based Classification Using a Multi-Task Convolutional Tsetlin Machine](https://mlsanity.com/paper/2510.01906/) (2025)

## Abstract (arXiv)

TMs are a pattern recognition approach that uses finite state machines for learning and propositional logic to represent patterns. In addition to being natively interpretable, they have provided competitive accuracy for various tasks. In this paper, we increase the computing power of TMs by proposing a first-order logic-based framework with Herbrand semantics. The resulting TM is relational and can take advantage of logical structures appearing in natural language, to learn rules that represent how actions and consequences are related in the real world. The outcome is a logic program of Horn clauses, bringing in a structured view of unstructured data. In closed-domain question-answering, the first-order representation produces 10x more compact KBs, along with an increase in answering accuracy from 94.83% to 99.48%. The approach is further robust towards erroneous, missing, and superfluous information, distilling the aspects of a text that are important for real-world understanding.