|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'506'133
Articles rated: 2609
26 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Robust Validation: Confident Predictions Even When Distributions Shift | | Maxime Cauchois
; Suyash Gupta
; Alnur Ali
; John C. Duchi
; | | Date: |
10 Aug 2020 | | Abstract: | While the traditional viewpoint in machine learning and statistics assumes
training and testing samples come from the same population, practice belies
this fiction. One strategy---coming from robust statistics and
optimization---is thus to build a model robust to distributional perturbations.
In this paper, we take a different approach to describe procedures for robust
predictive inference, where a model provides uncertainty estimates on its
predictions rather than point predictions. We present a method that produces
prediction sets (almost exactly) giving the right coverage level for any test
distribution in an $f$-divergence ball around the training population. The
method, based on conformal inference, achieves (nearly) valid coverage in
finite samples, under only the condition that the training data be
exchangeable. An essential component of our methodology is to estimate the
amount of expected future data shift and build robustness to it; we develop
estimators and prove their consistency for protection and validity of
uncertainty estimates under shifts. By experimenting on several large-scale
benchmark datasets, including Recht et al.’s CIFAR-v4 and ImageNet-V2 datasets,
we provide complementary empirical results that highlight the importance of
robust predictive validity. | | Source: | arXiv, 2008.04267 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER