| | |
| | |
Stat |
Members: 3643 Articles: 2'488'730 Articles rated: 2609
29 March 2024 |
|
| | | |
|
Article overview
| |
|
Kernel Alignment Risk Estimator: Risk Prediction from Training Data | Arthur Jacot
; Berfin Şimşek
; Francesco Spadaro
; Clément Hongler
; Franck Gabriel
; | Date: |
17 Jun 2020 | Abstract: | We study the risk (i.e. generalization error) of Kernel Ridge Regression
(KRR) for a kernel $K$ with ridge $lambda>0$ and i.i.d. observations. For
this, we introduce two objects: the Signal Capture Threshold (SCT) and the
Kernel Alignment Risk Estimator (KARE). The SCT $vartheta_{K,lambda}$ is a
function of the data distribution: it can be used to identify the components of
the data that the KRR predictor captures, and to approximate the (expected) KRR
risk. This then leads to a KRR risk approximation by the KARE $
ho_{K,
lambda}$, an explicit function of the training data, agnostic of the true data
distribution. We phrase the regression problem in a functional setting. The key
results then follow from a finite-size analysis of the Stieltjes transform of
general Wishart random matrices. Under a natural universality assumption (that
the KRR moments depend asymptotically on the first two moments of the
observations) we capture the mean and variance of the KRR predictor. We
numerically investigate our findings on the Higgs and MNIST datasets for
various classical kernels: the KARE gives an excellent approximation of the
risk, thus supporting our universality assumption. Using the KARE, one can
compare choices of Kernels and hyperparameters directly from the training set.
The KARE thus provides a promising data-dependent procedure to select Kernels
that generalize well. | Source: | arXiv, 2006.9796 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
| News, job offers and information for researchers and scientists:
| |