| | |
| | |
Stat |
Members: 3643 Articles: 2'487'895 Articles rated: 2609
28 March 2024 |
|
| | | |
|
Article overview
| |
|
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality | Yi Zhang
; Orestis Plevrakis
; Simon S. Du
; Xingguo Li
; Zhao Song
; Sanjeev Arora
; | Date: |
16 Feb 2020 | Abstract: | Adversarial training is a popular method to give neural nets robustness
against adversarial perturbations. In practice adversarial training leads to
low robust training loss. However, a rigorous explanation for why this happens
under natural conditions is still missing. Recently a convergence theory for
standard (non-adversarial) supervised training was developed by various groups
for {em very overparametrized} nets. It is unclear how to extend these results
to adversarial training because of the min-max objective. Recently, a first
step towards this direction was made by Gao et al. using tools from online
learning, but they require the width of the net to be emph{exponential} in
input dimension $d$, and with an unnatural activation function. Our work proves
convergence to low robust training loss for emph{polynomial} width instead of
exponential, under natural assumptions and with the ReLU activation. Key
element of our proof is showing that ReLU networks near initialization can
approximate the step function, which may be of independent interest. | Source: | arXiv, 2002.6668 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
| News, job offers and information for researchers and scientists:
| |