# ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model

arXiv: https://arxiv.org/abs/2009.03706 · HTML: https://mlsanity.com/paper/2009.03706/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2020.

## Built on by (most cited first, 3 of 3)

- [Learning to Emphasize: Dataset and Shared Task Models for Selecting Emphasis in Presentation Slides](https://mlsanity.com/paper/2101.03237/) (2021)
- [Stability-based Generalization Analysis for Mixtures of Pointwise and Pairwise Learning](https://mlsanity.com/paper/2302.09967/) (2023)
- [Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings](https://mlsanity.com/paper/2101.11422/) (2021)

## Abstract (arXiv)

This paper describes the system designed by ERNIE Team which achieved the first place in SemEval-2020 Task 10: Emphasis Selection For Written Text in Visual Media. Given a sentence, we are asked to find out the most important words as the suggestion for automated design. We leverage the unsupervised pre-training model and finetune these models on our task. After our investigation, we found that the following models achieved an excellent performance in this task: ERNIE 2.0, XLM-ROBERTA, ROBERTA and ALBERT. We combine a pointwise regression loss and a pairwise ranking loss which is more close to the final M atchm metric to finetune our models. And we also find that additional feature engineering and data augmentation can help improve the performance. Our best model achieves the highest score of 0.823 and ranks first for all kinds of metrics