# jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

arXiv: https://arxiv.org/abs/2003.02249 · HTML: https://mlsanity.com/paper/2003.02249/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 30 of 39)

- [On the Effectiveness of Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2211.15583/) (2022)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [Comparing Test Sets with Item Response Theory](https://mlsanity.com/paper/2106.00840/) (2021)
- [Language acquisition: do children and language models follow similar learning stages?](https://mlsanity.com/paper/2306.03586/) (2023)
- [An Analysis of Negation in Natural Language Understanding Corpora](https://mlsanity.com/paper/2203.08929/) (2022)
- [Curriculum learning for language modeling](https://mlsanity.com/paper/2108.02170/) (2021)
- [Hate Speech and Counter Speech Detection: Conversational Context Does Matter](https://mlsanity.com/paper/2206.06423/) (2022)
- [Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers](https://mlsanity.com/paper/2109.08406/) (2021)
- [Does Putting a Linguist in the Loop Improve NLU Data Collection?](https://mlsanity.com/paper/2104.07179/) (2021)
- [Investigating Reasons for Disagreement in Natural Language Inference](https://mlsanity.com/paper/2209.03392/) (2022)
- [BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models](https://mlsanity.com/paper/2404.02827/) (2024)
- [IndoNLI: A Natural Language Inference Dataset for Indonesian](https://mlsanity.com/paper/2110.14566/) (2021)
- [The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models](https://mlsanity.com/paper/2106.01950/) (2021)
- [LMdiff: A Visual Diff Tool to Compare Language Models](https://mlsanity.com/paper/2111.01582/) (2021)
- [Mediators in Determining what Processing BERT Performs First](https://mlsanity.com/paper/2104.06400/) (2021)
- [Machine Reading, Fast and Slow: When Do Models "Understand" Language?](https://mlsanity.com/paper/2209.07430/) (2022)
- [He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics](https://mlsanity.com/paper/2107.00807/) (2021)
- [Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair](https://mlsanity.com/paper/2111.08181/) (2021)
- [Hate Cannot Drive out Hate: Forecasting Conversation Incivility following Replies to Hate Speech](https://mlsanity.com/paper/2312.04804/) (2023)
- [Sequence to sequence pretraining for a less-resourced Slovenian language](https://mlsanity.com/paper/2207.13988/) (2022)
- [Paraphrasing in Affirmative Terms Improves Negation Understanding](https://mlsanity.com/paper/2406.07492/) (2024)
- [The Rediscovery Hypothesis: Language Models Need to Meet Linguistics](https://mlsanity.com/paper/2103.01819/) (2021)
- [Effect of Post-processing on Contextualized Word Representations](https://mlsanity.com/paper/2104.07456/) (2021)
- [Slovene SuperGLUE Benchmark: Translation and Evaluation](https://mlsanity.com/paper/2202.04994/) (2022)
- [Probing Linguistic Information For Logical Inference In Pre-trained Language Models](https://mlsanity.com/paper/2112.01753/) (2021)
- [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021)
- [ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models](https://mlsanity.com/paper/2205.01523/) (2022)
- [Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding](https://mlsanity.com/paper/2204.06283/) (2022)
- [Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?](https://mlsanity.com/paper/2109.07102/) (2021)
- [Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models](https://mlsanity.com/paper/2202.07791/) (2022)

## Abstract (arXiv)

We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimentation with state-of-the-art models and implements a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, including BERT and RoBERTa. jiant is available at https://jiant.info.