# Masked Sinogram Model with Transformer for ill-Posed Computed Tomography Reconstruction: a Preliminary Study

arXiv: https://arxiv.org/abs/2209.01356 · HTML: https://mlsanity.com/paper/2209.01356/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 3 Sep 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey of Deep Learning for Scientific Discovery](https://mlsanity.com/paper/2003.11755/) (2020)
- [TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks](https://mlsanity.com/paper/1902.07582/) (2019)
- [Bridging Data Center AI Systems with Edge Computing for Actionable Information Retrieval](https://mlsanity.com/paper/2105.13967/) (2021)
- [Deep Learning Accelerated Light Source Experiments](https://mlsanity.com/paper/1910.04081/) (2019)
- [Deep Learning-based Low-dose Tomography Reconstruction with Hybrid-dose Measurements](https://mlsanity.com/paper/2009.13589/) (2020)
- [BFTrainer: Low-Cost Training of Neural Networks on Unfillable Supercomputer Nodes](https://mlsanity.com/paper/2106.12091/) (2021)

## Built on by (most cited first, 2 of 2)

- [FCDM: A Physics-Guided Bidirectional Frequency Aware Convolution and Diffusion-Based Model for Sinogram Inpainting](https://mlsanity.com/paper/2409.06714/) (2024)
- [PRO: Projection Domain Synthesis for CT Imaging](https://mlsanity.com/paper/2506.13443/) (2025)

## Abstract (arXiv)

Computed Tomography (CT) is an imaging technique where information about an object are collected at different angles (called projections or scans). Then the cross-sectional image showing the internal structure of the slice is produced by solving an inverse problem. Limited by certain factors such as radiation dosage, projection angles, the produced images can be noisy or contain artifacts. Inspired by the success of transformer for natural language processing, the core idea of this preliminary study is to consider a projection of tomography as a word token, and the whole scan of the cross-section (A.K.A. sinogram) as a sentence in the context of natural language processing. Then we explore the idea of foundation model by training a masked sinogram model (MSM) and fine-tune MSM for various downstream applications including CT reconstruction under data collections restriction (e.g., photon-budget) and a data-driven solution to approximate solutions of the inverse problem for CT reconstruction. Models and data used in this study are available at https://github.com/lzhengchun/TomoTx.