# Word-level Human Interpretable Scoring Mechanism for Novel Text Detection Using Tsetlin Machines

arXiv: https://arxiv.org/abs/2105.04708 · HTML: https://mlsanity.com/paper/2105.04708/

## Key facts

- Cited by 10 papers in the map (data through 7 Oct 2026).
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://mlsanity.com/paper/1602.03483/) (2016)
- [The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic](https://mlsanity.com/paper/1804.01508/) (2018)
- [Generative Adversarial Active Learning for Unsupervised Outlier Detection](https://mlsanity.com/paper/1809.10816/) (2018)
- [Unsupervised Keyphrase Extraction with Multipartite Graphs](https://mlsanity.com/paper/1803.08721/) (2018)
- [The Convolutional Tsetlin Machine](https://mlsanity.com/paper/1905.09688/) (2019)
- [Open-Category Classification by Adversarial Sample Generation](https://mlsanity.com/paper/1705.08722/) (2017)
- [Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications](https://mlsanity.com/paper/1809.04547/) (2018)
- [On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators](https://mlsanity.com/paper/2007.14268/) (2020)
- [Massively Parallel and Asynchronous Tsetlin Machine Architecture Supporting Almost Constant-Time Scaling](https://mlsanity.com/paper/2009.04861/) (2020)
- [On the Convergence of Tsetlin Machines for the XOR Operator](https://mlsanity.com/paper/2101.02547/) (2021)
- [Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin Machine Text Classifier](https://mlsanity.com/paper/2011.08755/) (2020)
- [Enhancing Interpretable Clauses Semantically using Pretrained Word Representation](https://mlsanity.com/paper/2104.06901/) (2021)
- [Novelty Detection with GAN](https://mlsanity.com/paper/1802.10560/) (2018)
- [The Regression Tsetlin Machine: A Tsetlin Machine for Continuous Output Problems](https://mlsanity.com/paper/1905.04206/) (2019)

## Built on by (most cited first, 10 of 10)

- [Explainable Tsetlin Machine framework for fake news detection with credibility score assessment](https://mlsanity.com/paper/2105.09114/) (2021)
- [Coalesced Multi-Output Tsetlin Machines with Clause Sharing](https://mlsanity.com/paper/2108.07594/) (2021)
- [Tsetlin Machine Embedding: Representing Words Using Logical Expressions](https://mlsanity.com/paper/2301.00709/) (2023)
- [Building Concise Logical Patterns by Constraining Tsetlin Machine Clause Size](https://mlsanity.com/paper/2301.08190/) (2023)
- [Exploring State Space and Reasoning by Elimination in Tsetlin Machines](https://mlsanity.com/paper/2407.09162/) (2024)
- [Scalable Multi-phase Word Embedding Using Conjunctive Propositional Clauses](https://mlsanity.com/paper/2501.19018/) (2025)
- [Contracting Tsetlin Machine with Absorbing Automata](https://mlsanity.com/paper/2310.11481/) (2023)
- [The Sparse Tsetlin Machine: Sparse Representation with Active Literals](https://mlsanity.com/paper/2405.02375/) (2024)
- [Generalized Convergence Analysis of Tsetlin Machines: A Probabilistic Approach to Concept Learning](https://mlsanity.com/paper/2310.02005/) (2023)
- [Pruning Literals for Highly Efficient Explainability at Word Level](https://mlsanity.com/paper/2411.04557/) (2024)

## Abstract (arXiv)

Recent research in novelty detection focuses mainly on document-level classification, employing deep neural networks (DNN). However, the black-box nature of DNNs makes it difficult to extract an exact explanation of why a document is considered novel. In addition, dealing with novelty at the word-level is crucial to provide a more fine-grained analysis than what is available at the document level. In this work, we propose a Tsetlin machine (TM)-based architecture for scoring individual words according to their contribution to novelty. Our approach encodes a description of the novel documents using the linguistic patterns captured by TM clauses. We then adopt this description to measure how much a word contributes to making documents novel. Our experimental results demonstrate how our approach breaks down novelty into interpretable phrases, successfully measuring novelty.