# Semantically Enriched Cross-Lingual Sentence Embeddings for Crisis-related Social Media Texts

arXiv: https://arxiv.org/abs/2403.16614 · HTML: https://mlsanity.com/paper/2403.16614/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [Multilingual Universal Sentence Encoder for Semantic Retrieval](https://mlsanity.com/paper/1907.04307/) (2019)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://mlsanity.com/paper/1602.03483/) (2016)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [Processing Social Media Messages in Mass Emergency: A Survey](https://mlsanity.com/paper/1407.7071/) (2014)
- [Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages](https://mlsanity.com/paper/1605.05894/) (2016)
- [Learning Semantic Textual Similarity from Conversations](https://mlsanity.com/paper/1804.07754/) (2018)
- [Did You Really Just Have a Heart Attack? Towards Robust Detection of Personal Health Mentions in Social Media](https://mlsanity.com/paper/1802.09130/) (2018)
- [Winds of Change: Impact of COVID-19 on Vaccine-related Opinions of Twitter users](https://mlsanity.com/paper/2111.10667/) (2021)
- [A Twitter Tale of Three Hurricanes: Harvey, Irma, and Maria](https://mlsanity.com/paper/1805.05144/) (2018)
- [Utilizing Microblogs for Assisting Post-Disaster Relief Operations via Matching Resource Needs and Availabilities](https://mlsanity.com/paper/2007.09368/) (2020)
- [CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts](https://mlsanity.com/paper/2309.05494/) (2023)
- [Where did you tweet from? Inferring the origin locations of tweets based on contextual information](https://mlsanity.com/paper/2211.16506/) (2022)
- [Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey](https://mlsanity.com/paper/2209.07272/) (2022)
- [A Twitter narrative of the COVID-19 pandemic in Australia](https://mlsanity.com/paper/2302.11136/) (2023)

## Built on by (most cited first, 2 of 2)

- [CReMa: Crisis Response through Computational Identification and Matching of Cross-Lingual Requests and Offers Shared on Social Media](https://mlsanity.com/paper/2405.11897/) (2024)
- [IfcLLM: Natural Language Querying of IFC Models through Complementary Relational and Graph Representations](https://mlsanity.com/paper/2605.13236/) (2026)

## Abstract (arXiv)

Tasks such as semantic search and clustering on crisis-related social media texts enhance our comprehension of crisis discourse, aiding decision-making and targeted interventions. Pre-trained language models have advanced performance in crisis informatics, but their contextual embeddings lack semantic meaningfulness. Although the CrisisTransformers family includes a sentence encoder to address the semanticity issue, it remains monolingual, processing only English texts. Furthermore, employing separate models for different languages leads to embeddings in distinct vector spaces, introducing challenges when comparing semantic similarities between multi-lingual texts. Therefore, we propose multi-lingual sentence encoders (CT-XLMR-SE and CT-mBERT-SE) that embed crisis-related social media texts for over 50 languages, such that texts with similar meanings are in close proximity within the same vector space, irrespective of language diversity. Results in sentence encoding and sentence matching tasks are promising, suggesting these models could serve as robust baselines when embedding multi-lingual crisis-related social media texts. The models are publicly available at: https://huggingface.co/crisistransformers.