# A Layer-Wise Tokens-to-Token Transformer Network for Improved Historical Document Image Enhancement

arXiv: https://arxiv.org/abs/2312.03946 · HTML: https://mlsanity.com/paper/2312.03946/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [DE-GAN: A Conditional Generative Adversarial Network for Document Enhancement](https://mlsanity.com/paper/2010.08764/) (2020)
- [DocEnTr: An End-to-End Document Image Enhancement Transformer](https://mlsanity.com/paper/2201.10252/) (2022)
- [Document Image Binarization with Fully Convolutional Neural Networks](https://mlsanity.com/paper/1708.03276/) (2017)
- [Polyp-SAM++: Can A Text Guided SAM Perform Better for Polyp Segmentation?](https://mlsanity.com/paper/2308.06623/) (2023)
- [Improving Document Binarization via Adversarial Noise-Texture Augmentation](https://mlsanity.com/paper/1810.11120/) (2018)
- [Two-stage generative adversarial networks for document image binarization with color noise and background removal](https://mlsanity.com/paper/2010.10103/) (2020)
- [SauvolaNet: Learning Adaptive Sauvola Network for Degraded Document Binarization](https://mlsanity.com/paper/2105.05521/) (2021)

## Built on by (most cited first, 2 of 2)

- [Unveiling Text in Challenging Stone Inscriptions: A Character-Context-Aware Patching Strategy for Binarization](https://mlsanity.com/paper/2601.03609/) (2026)
- [Integrated Framework for Selecting and Enhancing Ancient Marathi Inscription Images from Stone, Metal Plate, and Paper Documents](https://mlsanity.com/paper/2601.04800/) (2026)

## Abstract (arXiv)

Document image enhancement is a fundamental and important stage for attaining the best performance in any document analysis assignment because there are many degradation situations that could harm document images, making it more difficult to recognize and analyze them. In this paper, we propose T2T-BinFormer which is a novel document binarization encoder-decoder architecture based on a Tokens-to-token vision transformer. Each image is divided into a set of tokens with a defined length using the ViT model, which is then applied several times to model the global relationship between the tokens. However, the conventional tokenization of input data does not adequately reflect the crucial local structure between adjacent pixels of the input image, which results in low efficiency. Instead of using a simple ViT and hard splitting of images for the document image enhancement task, we employed a progressive tokenization technique to capture this local information from an image to achieve more effective results. Experiments on various DIBCO and H-DIBCO benchmarks demonstrate that the proposed model outperforms the existing CNN and ViT-based state-of-the-art methods. In this research, the primary area of examination is the application of the proposed architecture to the task of document binarization. The source code will be made available at https://github.com/RisabBiswas/T2T-BinFormer.