| | |
| | |
Stat |
Members: 3643 Articles: 2'487'895 Articles rated: 2609
28 March 2024 |
|
| | | |
|
Article overview
| |
|
Language GANs Falling Short | Massimo Caccia
; Lucas Caccia
; William Fedus
; Hugo Larochelle
; Joelle Pineau
; Laurent Charlin
; | Date: |
8 Nov 2018 | Abstract: | Generating high-quality text with sufficient diversity is essential for a
wide range of Natural Language Generation (NLG) tasks. Maximum-Likelihood (MLE)
models trained with teacher forcing have constantly been reported as weak
baselines, where poor performance is attributed to exposure bias; at inference
time, the model is fed its own prediction instead of a ground-truth token,
which can lead to accumulating errors and poor samples. This line of reasoning
has led to an outbreak of adversarial based approaches for NLG, on the account
that GANs do not suffer from exposure bias. In this work, we make several
surprising observations with contradict common beliefs. We first revisit the
canonical evaluation framework for NLG, and point out fundamental flaws with
quality-only evaluation: we show that one can outperform such metrics using a
simple, well-known temperature parameter to artificially reduce the entropy of
the model’s conditional distributions. Second, we leverage the control over the
quality / diversity tradeoff given by this parameter to evaluate models over
the whole quality-diversity spectrum, and find MLE models constantly outperform
the proposed GAN variants, over the whole quality-diversity space. Our results
have several implications: 1) The impact of exposure bias on sample quality is
less severe than previously thought, 2) temperature tuning provides a better
quality / diversity trade off than adversarial training, while being easier to
train, easier to cross-validate, and less computationally expensive. | Source: | arXiv, 1811.2549 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
| News, job offers and information for researchers and scientists:
| |