# MeetDot: Videoconferencing with Live Translation Captions

arXiv: https://arxiv.org/abs/2109.09577 · HTML: https://mlsanity.com/paper/2109.09577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Low-Latency Neural Speech Translation](https://mlsanity.com/paper/1808.00491/) (2018)
- [Re-Translation Strategies For Long Form, Simultaneous, Spoken Language Translation](https://mlsanity.com/paper/1912.03393/) (2019)
- [Simultaneous Translation with Flexible Policy via Restricted Imitation Learning](https://mlsanity.com/paper/1906.01135/) (2019)
- [DiDi's Machine Translation System for WMT2020](https://mlsanity.com/paper/2010.08185/) (2020)

## Built on by (most cited first, 1 of 1)

- [LINC: Supporting Language Independent Communication and Comprehension to Enhance Contribution in Multilingual Collaborative Meetings](https://mlsanity.com/paper/2504.18988/) (2025)

## Abstract (arXiv)

We present MeetDot, a videoconferencing system with live translation captions overlaid on screen. The system aims to facilitate conversation between people who speak different languages, thereby reducing communication barriers between multilingual participants. Currently, our system supports speech and captions in 4 languages and combines automatic speech recognition (ASR) and machine translation (MT) in a cascade. We use the re-translation strategy to translate the streamed speech, resulting in caption flicker. Additionally, our system has very strict latency requirements to have acceptable call quality. We implement several features to enhance user experience and reduce their cognitive load, such as smooth scrolling captions and reducing caption flicker. The modular architecture allows us to integrate different ASR and MT services in our backend. Our system provides an integrated evaluation suite to optimize key intrinsic evaluation metrics such as accuracy, latency and erasure. Finally, we present an innovative cross-lingual word-guessing game as an extrinsic evaluation metric to measure end-to-end system performance. We plan to make our system open-source for research purposes.