|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'506'133
Articles rated: 2609
26 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval | | Stefano Sarao Mannelli
; Giulio Biroli
; Chiara Cammarota
; Florent Krzakala
; Pierfrancesco Urbani
; Lenka Zdeborová
; | | Date: |
12 Jun 2020 | | Abstract: | Despite the widespread use of gradient-based algorithms for optimizing
high-dimensional non-convex functions, understanding their ability of finding
good minima instead of being trapped in spurious ones remains to a large extent
an open problem. Here we focus on gradient flow dynamics for phase retrieval
from random measurements. When the ratio of the number of measurements over the
input dimension is small the dynamics remains trapped in spurious minima with
large basins of attraction. We find analytically that above a critical ratio
those critical points become unstable developing a negative direction toward
the signal. By numerical experiments we show that in this regime the gradient
flow algorithm is not trapped; it drifts away from the spurious critical points
along the unstable direction and succeeds in finding the global minimum. Using
tools from statistical physics we characterize this phenomenon, which is
related to a BBP-type transition in the Hessian of the spurious minima. | | Source: | arXiv, 2006.6997 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER