mlsanityData to 7 Oct 2026

Cedille: A large autoregressive French language model

arXiv 2202.03371 ↗First posted 7 Feb 2022cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Scaling up the size and training of autoregressive language models has enabled novel ways of solving Natural Language Processing tasks using zero-shot and few-shot learning. While extreme-scale language models such as GPT-3 offer multilingual capabilities, zero-shot learning for languages other than English remain largely unexplored. Here, we introduce Cedille, a large open source auto-regressive language model, specifically trained for the French language. Our results show that Cedille outperforms existing French language models and is competitive with GPT-3 on a range of French zero-shot benchmarks. Furthermore, we provide an in-depth comparison of the toxicity exhibited by these models, showing that Cedille marks an improvement in language model safety thanks to dataset filtering.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

24 of 38 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  10. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  12. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
12 more
  1. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  2. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  3. 2019CamemBERT: a Tasty French Language Modelcited by 368
  4. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  5. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  6. 2021Challenges in Detoxifying Language Modelscited by 169
  7. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  8. 2020FQuAD: French Question Answering Datasetcited by 58
  9. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  10. 2020Very Deep Transformers for Neural Machine Translationcited by 50
  11. 2020BARThez: a Skilled Pretrained French Sequence-to-Sequence Modelcited by 39
  12. 2021PAGnol: An Extra-Large French Generative Modelcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 7; 2025: 7; 2026: 1.

20 papers.

Sort:

By area:

  1. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  2. 2023Typhoon: Thai Large Language Modelscited by 28
  3. 2024Exploring Design Choices for Building Language-Specific LLMscited by 22
  4. 2022Domain-Specific Text Generation for Machine Translationcited by 12
  5. 2024Large Vocabulary Size Improves Large Language Modelscited by 11
  6. 2023Data-Efficient French Language Modeling with CamemBERTacited by 8
  7. 2025Overcoming Data Scarcity in Generative Language Modelling for Low-Resource Languages: A Systematic Reviewcited by 8
  8. 2024A Survey of Large Language Models for European Languagescited by 6
  9. 2022Effectiveness of French Language Models on Abstractive Dialogue Summarization Taskcited by 3
  10. 2024Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languagescited by 3
  11. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  12. 2026Pantagruel: Unified Self-Supervised Encoders for French Text and Speechcited by 2
8 more
  1. 2024LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentationcited by 1
  2. 2024Know When to Fuse: Investigating Non-English Hybrid Retrieval in the Legal Domaincited by 1
  3. 2025Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Modelscited by 0
  4. 2025Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Modelscited by 0
  5. 2025Multi-granular Training Strategies for Robust Multi-hop Reasoning Over Noisy and Heterogeneous Knowledge Sourcescited by 0
  6. 2025LakotaBERT: A Transformer-based Model for Low Resource Lakota Languagecited by 0
  7. 2025Leveraging Vision-Language Large Models for Interpretable Video Action Recognition with Semantic Tokenizationcited by 0
  8. 2025Enhancing Factual Accuracy and Citation Generation in LLMs via Multi-Stage Self-Verificationcited by 0