mlsanityData to 7 Oct 2026

MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki

arXiv 2403.07544 ↗First posted 12 Mar 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

NLP in the age of monolithic large language models is approaching its limits in terms of size and information that can be handled. The trend goes to modularization, a necessary step into the direction of designing smaller sub-networks and components with specialized functionality. In this paper, we present the MAMMOTH toolkit: a framework designed for training massively multilingual modular machine translation systems at scale, initially derived from OpenNMT-py and then adapted to ensure efficient training across computation clusters. We showcase its efficiency across clusters of A100 and V100 NVIDIA GPUs, and discuss our design philosophy and plans for future information. The toolkit is publicly available online.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

14 of 32 references.

References · 14

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  8. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  9. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  10. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  11. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  12. 2020Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoderscited by 23
2 more
  1. 2022Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translationcited by 21
  2. 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025What Are They Filtering Out? An Experimental Benchmark of Filtering Strategies for Harm Reduction in Pretraining Datasetscited by 1
  2. 2024I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architecturescited by 0