# Sinhala Sentence Embedding: A Two-Tiered Structure for Low-Resource Languages

arXiv: https://arxiv.org/abs/2210.14472 · HTML: https://mlsanity.com/paper/2210.14472/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [A Primer on Neural Network Models for Natural Language Processing](https://mlsanity.com/paper/1510.00726/) (2015)
- [Embedding Text in Hyperbolic Spaces](https://mlsanity.com/paper/1806.04313/) (2018)
- [Neural Embeddings of Graphs in Hyperbolic Space](https://mlsanity.com/paper/1705.10359/) (2017)
- [Survey on Publicly Available Sinhala Natural Language Processing Tools and Research](https://mlsanity.com/paper/1906.02358/) (2019)
- [Investigating Capsule Networks with Dynamic Routing for Text Classification](https://mlsanity.com/paper/1804.00538/) (2018)
- [Skip-gram word embeddings in hyperbolic space](https://mlsanity.com/paper/1809.01498/) (2018)
- [Distant supervision for emotion detection using Facebook reactions](https://mlsanity.com/paper/1611.02988/) (2016)
- [Sentiment Analysis for Sinhala Language using Deep Learning Techniques](https://mlsanity.com/paper/2011.07280/) (2020)
- [Measuring the Diversity of Facebook Reactions to Research](https://mlsanity.com/paper/2001.01029/) (2020)
- [Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook](https://mlsanity.com/paper/2007.07884/) (2020)
- [Geometry of Polysemy](https://mlsanity.com/paper/1610.07569/) (2016)
- [Jointly Learning to Detect Emotions and Predict Facebook Reactions](https://mlsanity.com/paper/1909.10779/) (2019)
- [Seeking Sinhala Sentiment: Predicting Facebook Reactions of Sinhala Posts](https://mlsanity.com/paper/2112.00468/) (2021)
- [Sentiment Analysis with Deep Learning Models: A Comparative Study on a Decade of Sinhala Language Facebook Data](https://mlsanity.com/paper/2201.03941/) (2022)

## Built on by (most cited first, 2 of 2)

- [Translation Aligned Sentence Embeddings for Turkish Language](https://mlsanity.com/paper/2311.09748/) (2023)
- [Fault of Our Stars: Behavioral Drivers of Rating-Sentiment Incongruence](https://mlsanity.com/paper/2606.25518/) (2026)

## Abstract (arXiv)

In the process of numerically modeling natural languages, developing language embeddings is a vital step. However, it is challenging to develop functional embeddings for resource-poor languages such as Sinhala, for which sufficiently large corpora, effective language parsers, and any other required resources are difficult to find. In such conditions, the exploitation of existing models to come up with an efficacious embedding methodology to numerically represent text could be quite fruitful. This paper explores the effectivity of several one-tiered and two-tiered embedding architectures in representing Sinhala text in the sentiment analysis domain. With our findings, the two-tiered embedding architecture where the lower-tier consists of a word embedding and the upper-tier consists of a sentence embedding has been proven to perform better than one-tier word embeddings, by achieving a maximum F1 score of 88.04% in contrast to the 83.76% achieved by word embedding models. Furthermore, embeddings in the hyperbolic space are also developed and compared with Euclidean embeddings in terms of performance. A sentiment data set consisting of Facebook posts and associated reactions have been used for this research. To effectively compare the performance of different embedding systems, the same deep neural network structure has been trained on sentiment data with each of the embedding systems used to encode the text associated.