|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'501'711
Articles rated: 2609
20 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
On the Expressive Power of Overlapping Operations of Deep Networks | | Or Sharir
; Amnon Shashua
; | | Date: |
6 Mar 2017 | | Abstract: | Expressive Efficiency with respect to a network architectural attribute P
refers to the property where an architecture without P must grow exponentially
large in order to approximate the expressivity of a network with attribute P.
For example, it is known that depth is an architectural attribute that
generates exponential efficiency in the sense that a shallow network must grow
exponentially large in order to approximate the functions represented by a deep
network of polynomial size. In this paper we extend the study of expressive
efficiency to the attribute of network connectivity and in particular to the
effect of "overlaps" in the convolutional process, i.e., when the stride of the
convolution is smaller than its kernel size (receptive field).
Our analysis shows that having overlapping local receptive fields, and more
broadly denser connectivity, results in an exponential increase in the
expressive capacity of neural networks. Moreover, while denser connectivity can
increase the expressive capacity, we show that the most common types of modern
architectures already exhibit exponential increase in expressivity, without
relying on fully-connected layers. | | Source: | arXiv, 1703.2065 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER