# NLP Meets RNA: Unsupervised Embedding Learning for Ribozymes with Word2Vec

arXiv: https://arxiv.org/abs/2307.05537 · HTML: https://mlsanity.com/paper/2307.05537/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 8 Jul 2023.

## Builds on (most cited first, 7 of 7)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Word and Phrase Translation with word2vec](https://mlsanity.com/paper/1705.03127/) (2017)
- [Word2Vec: Optimal Hyper-Parameters and Their Impact on NLP Downstream Tasks](https://mlsanity.com/paper/2003.11645/) (2020)
- [Sentiment Analysis of Citations Using Word2vec](https://mlsanity.com/paper/1704.00177/) (2017)
- [Vec2Vec: A Compact Neural Network Approach for Transforming Text Embeddings with High Fidelity](https://mlsanity.com/paper/2306.12689/) (2023)

## Abstract (arXiv)

Ribozymes, RNA molecules with distinct 3D structures and catalytic activity, have widespread applications in synthetic biology and therapeutics. However, relatively little research has focused on leveraging deep learning to enhance our understanding of ribozymes. This study implements Word2Vec, an unsupervised learning technique for natural language processing, to learn ribozyme embeddings. Ribo2Vec was trained on over 9,000 diverse ribozymes, learning to map sequences to 128 and 256-dimensional vector spaces. Using Ribo2Vec, sequence embeddings for five classes of ribozymes (hatchet, pistol, hairpin, hovlinc, and twister sister) were calculated. Principal component analysis demonstrated the ability of these embeddings to distinguish between ribozyme classes. Furthermore, a simple SVM classifier trained on ribozyme embeddings showed promising results in accurately classifying ribozyme types. Our results suggest that the embedding vectors contained meaningful information about ribozymes. Interestingly, 256-dimensional embeddings behaved similarly to 128-dimensional embeddings, suggesting that a lower dimension vector space is generally sufficient to capture ribozyme features. This approach demonstrates the potential of Word2Vec for bioinformatics, opening new avenues for ribozyme research. Future research includes using a Transformer-based method to learn RNA embeddings, which can capture long-range interactions between nucleotides.