# Harvesting comparable corpora and mining them for equivalent bilingual sentences using statistical classification and analogy- based heuristics

arXiv: https://arxiv.org/abs/1511.06285 · HTML: https://mlsanity.com/paper/1511.06285/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2015.

## Built on by (most cited first, 1 of 1)

- [Data Augmentation for Code Translation with Comparable Corpora and Multiple References](https://mlsanity.com/paper/2311.00317/) (2023)

## Abstract (arXiv)

Parallel sentences are a relatively scarce but extremely useful resource for many applications including cross-lingual retrieval and statistical machine translation. This research explores our new methodologies for mining such data from previously obtained comparable corpora. The task is highly practical since non-parallel multilingual data exist in far greater quantities than parallel corpora, but parallel sentences are a much more useful resource. Here we propose a web crawling method for building subject-aligned comparable corpora from e.g. Wikipedia dumps and Euronews web page. The improvements in machine translation are shown on Polish-English language pair for various text domains. We also tested another method of building parallel corpora based on comparable corpora data. It lets automatically broad existing corpus of sentences from subject of corpora based on analogies between them.