On Training Bi-directional Neural Network Language Model with Noise Contrastive Estimation
Not on the map. Why?
Abstract (arXiv)
We propose to train bi-directional neural network language model(NNLM) with noise contrastive estimation(NCE). Experiments are conducted on a rescore task on the PTB data set. It is shown that NCE-trained bi-directional NNLM outperformed the one trained by conventional maximum likelihood training. But still(regretfully), it did not out-perform the baseline uni-directional NNLM.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Feb 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3.
3 papers.
- 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognitioncited by 16
- 2021Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Modelscited by 14
- 2021Learning a Word-Level Language Model with Sentence-Level Noise Contrastive Estimation for Contextual Sentence Probability Estimationcited by 0
- 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition29 Jul 2021
- 2021Learning a Word-Level Language Model with Sentence-Level Noise Contrastive Estimation for Contextual Sentence Probability Estimation14 Mar 2021
- 2021Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models18 Jan 2021