# Two-Stage COVID19 Classification Using BERT Features

arXiv: https://arxiv.org/abs/2206.14861 · HTML: https://mlsanity.com/paper/2206.14861/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Deep Transparent Prediction through Latent Representation Analysis](https://mlsanity.com/paper/2009.07044/) (2020)
- [MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis](https://mlsanity.com/paper/2106.07524/) (2021)
- [Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans](https://mlsanity.com/paper/2101.05442/) (2021)
- [A 3D CNN Network with BERT For Automatic COVID-19 Diagnosis From CT-Scan Images](https://mlsanity.com/paper/2106.14403/) (2021)
- [COVID-MTL: Multitask Learning with Shift3D and Random-weighted Loss for Automated Diagnosis and Severity Assessment of COVID-19](https://mlsanity.com/paper/2012.05509/) (2020)

## Built on by (most cited first, 2 of 2)

- [COVID-19 Detection Using Slices Processing Techniques and a Modified Xception Classifier from Computed Tomography Images](https://mlsanity.com/paper/2312.07580/) (2023)
- [COVID-19 Detection Using Transfer Learning Approach from Computed Tomography Images](https://mlsanity.com/paper/2207.00259/) (2022)

## Abstract (arXiv)

We propose an automatic COVID1-19 diagnosis framework from lung CT-scan slice images using double BERT feature extraction. In the first BERT feature extraction, A 3D-CNN is first used to extract CNN internal feature maps. Instead of using the global average pooling, a late BERT temporal pooing is used to aggregate the temporal information in these feature maps, followed by a classification layer. This 3D-CNN-BERT classification network is first trained on sampled fixed number of slice images from every original CT scan volume. In the second stage, the 3D-CNN-BERT embedding features are extracted on all slice images of every CT scan volume, and these features are averaged into a fixed number of segments. Then another BERT network is used to aggregate these multiple features into a single feature followed by another classification layer. The classification results of both stages are combined to generate final outputs. On the validation dataset, we achieve macro F1 score of 0.9164.