# SemanticCAP: Chromatin Accessibility Prediction Enhanced by Features Learning from a Language Model

arXiv: https://arxiv.org/abs/2204.02130 · HTML: https://mlsanity.com/paper/2204.02130/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 41 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Gated End-to-End Memory Networks](https://mlsanity.com/paper/1610.04211/) (2016)

## Built on by (most cited first, 2 of 2)

- [Deciphering genomic codes using advanced NLP techniques: a scoping review](https://mlsanity.com/paper/2411.16084/) (2024)
- [Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees](https://mlsanity.com/paper/2608.02455/) (2026)

## Abstract (arXiv)

A large number of inorganic and organic compounds are able to bind DNA and form complexes, among which drug-related molecules are important. Chromatin accessibility changes not only directly affects drug-DNA interactions, but also promote or inhibit the expression of critical genes associated with drug resistance by affecting the DNA binding capacity of TFs and transcriptional regulators. However, Biological experimental techniques for measuring it are expensive and time consuming. In recent years, several kinds of computational methods have been proposed to identify accessible regions of the genome. Existing computational models mostly ignore the contextual information of bases in gene sequences. To address these issues, we proposed a new solution named SemanticCAP. It introduces a gene language model which models the context of gene sequences, thus being able to provide an effective representation of a certain site in gene sequences. Basically, we merge the features provided by the gene language model into our chromatin accessibility model. During the process, we designed some methods to make feature fusion smoother. Compared with other systems under public benchmarks, our model proved to have better performance.