# End-to-End Lip Reading in Romanian with Cross-Lingual Domain Adaptation and Lateral Inhibition

arXiv: https://arxiv.org/abs/2310.04858 · HTML: https://mlsanity.com/paper/2310.04858/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [Lipreading using Temporal Convolutional Networks](https://mlsanity.com/paper/2001.08702/) (2020)
- [Combining Residual Networks with LSTMs for Lipreading](https://mlsanity.com/paper/1703.04105/) (2017)
- [Sub-word Level Lip Reading With Visual Attention](https://mlsanity.com/paper/2110.07603/) (2021)
- [Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks](https://mlsanity.com/paper/2009.08378/) (2020)
- [LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild](https://mlsanity.com/paper/1810.06990/) (2018)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Learn an Effective Lip Reading Model without Pains](https://mlsanity.com/paper/2011.07557/) (2020)
- [Lipreading with Long Short-Term Memory](https://mlsanity.com/paper/1601.08188/) (2016)
- [Speaker-adaptive Lip Reading with User-dependent Padding](https://mlsanity.com/paper/2208.04498/) (2022)
- [A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning](https://mlsanity.com/paper/2202.13403/) (2022)
- [Advances and Challenges in Deep Lip Reading](https://mlsanity.com/paper/2110.07879/) (2021)
- [Personalized One-Shot Lipreading for an ALS Patient](https://mlsanity.com/paper/2111.01740/) (2021)

## Built on by (most cited first, 1 of 1)

- [VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness](https://mlsanity.com/paper/2607.08112/) (2026)

## Abstract (arXiv)

Lip reading or visual speech recognition has gained significant attention in recent years, particularly because of hardware development and innovations in computer vision. While considerable progress has been obtained, most models have only been tested on a few large-scale datasets. This work addresses this shortcoming by analyzing several architectures and optimizations on the underrepresented, short-scale Romanian language dataset called Wild LRRo. Most notably, we compare different backend modules, demonstrating the effectiveness of adding ample regularization methods. We obtain state-of-the-art results using our proposed method, namely cross-lingual domain adaptation and unlabeled videos from English and German datasets to help the model learn language-invariant features. Lastly, we assess the performance of adding a layer inspired by the neural inhibition mechanism.