# VizSeq: A Visual Analysis Toolkit for Text Generation Tasks

arXiv: https://arxiv.org/abs/1909.05424 · HTML: https://mlsanity.com/paper/1909.05424/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2019.

## Built on by (most cited first, 10 of 10)

- [fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit](https://mlsanity.com/paper/2109.06912/) (2021)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts](https://mlsanity.com/paper/2403.04758/) (2024)
- [iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries](https://mlsanity.com/paper/2403.04760/) (2024)
- [Zero-shot Speech Translation](https://mlsanity.com/paper/2107.06010/) (2021)
- [Summary Explorer: Visualizing the State of the Art in Text Summarization](https://mlsanity.com/paper/2108.01879/) (2021)
- [Tackling data scarcity in speech translation using zero-shot multilingual machine translation techniques](https://mlsanity.com/paper/2201.11172/) (2022)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [Efficient Speech Translation with Pre-trained Models](https://mlsanity.com/paper/2211.04939/) (2022)
- [string2string Studio: An Interactive, In-Browser Platform for String-to-String Algorithms](https://mlsanity.com/paper/2608.03984/) (2026)

## Abstract (arXiv)

Automatic evaluation of text generation tasks (e.g. machine translation, text summarization, image captioning and video description) usually relies heavily on task-specific metrics, such as BLEU and ROUGE. They, however, are abstract numbers and are not perfectly aligned with human assessment. This suggests inspecting detailed examples as a complement to identify system error patterns. In this paper, we present VizSeq, a visual analysis toolkit for instance-level and corpus-level system evaluation on a wide variety of text generation tasks. It supports multimodal sources and multiple text references, providing visualization in Jupyter notebook or a web app interface. It can be used locally or deployed onto public servers for centralized data hosting and benchmarking. It covers most common n-gram based metrics accelerated with multiprocessing, and also provides latest embedding-based metrics such as BERTScore.