|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'928
Articles rated: 2609
26 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Boosting Dilated Convolutional Networks with Mixed Tensor Decompositions | | Nadav Cohen
; Ronen Tamari
; Amnon Shashua
; | | Date: |
20 Mar 2017 | | Abstract: | Expressive efficiency is a concept that allows formally reasoning about the
representational capacity of deep network architectures. A network architecture
is expressively efficient with respect to an alternative architecture if the
latter must grow super-linearly in order to represent functions realized by the
former. A well-known example is the exponential expressive efficiency of depth,
namely, that in many cases shallow networks must grow exponentially large in
order to represent functions realized by deep networks. In this paper we study
the expressive efficiency brought forth by the architectural feature of
connectivity, motivated by the observation that nearly all state of the art
networks these days employ elaborate connection schemes, running layers in
parallel while splitting and merging them in various ways. A formal treatment
of this question would shed light on the effectiveness of modern connectivity
schemes, and in addition, could provide new tools for network design. We focus
on dilated convolutional networks, a family of deep models gaining increased
attention, underlying state of the art architectures like Google’s WaveNet and
ByteNet. By introducing and studying the concept of mixed tensor
decompositions, we prove that interconnecting dilated convolutional networks
can lead to expressive efficiency. In particular, we show that a single
connection between intermediate layers can already lead to an almost quadratic
gap, which in large-scale settings typically makes the difference between a
model that is practical and one that is not. | | Source: | arXiv, 1703.6846 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER