|
| | | | |
| | | |
| Stat |
Members: 3644
Articles: 2'497'992
Articles rated: 2609
16 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift | | Yaniv Ovadia
; Emily Fertig
; Jie Ren
; Zachary Nado
; D Sculley
; Sebastian Nowozin
; Joshua V. Dillon
; Balaji Lakshminarayanan
; Jasper Snoek
; | | Date: |
6 Jun 2019 | | Abstract: | Modern machine learning methods including deep learning have achieved great
success in predictive accuracy for supervised learning tasks, but may still
fall short in giving useful estimates of their predictive {em uncertainty}.
Quantifying uncertainty is especially critical in real-world settings, which
often involve input distributions that are shifted from the training
distribution due to a variety of factors including sample bias and
non-stationarity. In such settings, well calibrated uncertainty estimates
convey information about when a model’s output should (or should not) be
trusted. Many probabilistic deep learning methods, including Bayesian-and
non-Bayesian methods, have been proposed in the literature for quantifying
predictive uncertainty, but to our knowledge there has not previously been a
rigorous large-scale empirical comparison of these methods under dataset shift.
We present a large-scale benchmark of existing state-of-the-art methods on
classification problems and investigate the effect of dataset shift on accuracy
and calibration. We find that traditional post-hoc calibration does indeed fall
short, as do several other previous methods. However, some methods that
marginalize over models give surprisingly strong results across a broad
spectrum of tasks. | | Source: | arXiv, 1906.2530 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER