| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
09 February 2025 |
|
| | | |
|
Article overview
| |
|
Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption | Matthias Mitterreiter
; Marcel Koch
; Joachim Giesen
; Sören Laue
; | Date: |
4 Jan 2023 | Abstract: | Capsule neural networks replace simple, scalar-valued neurons with
vector-valued capsules. They are motivated by the pattern recognition system in
the human brain, where complex objects are decomposed into a hierarchy of
simpler object parts. Such a hierarchy is referred to as a parse-tree.
Conceptually, capsule neural networks have been defined to realize such
parse-trees. The capsule neural network (CapsNet), by Sabour, Frosst, and
Hinton, is the first actual implementation of the conceptual idea of capsule
neural networks. CapsNets achieved state-of-the-art performance on simple image
recognition tasks with fewer parameters and greater robustness to affine
transformations than comparable approaches. This sparked extensive follow-up
research. However, despite major efforts, no work was able to scale the CapsNet
architecture to more reasonable-sized datasets. Here, we provide a reason for
this failure and argue that it is most likely not possible to scale CapsNets
beyond toy examples. In particular, we show that the concept of a parse-tree,
the main idea behind capsule neuronal networks, is not present in CapsNets. We
also show theoretically and experimentally that CapsNets suffer from a
vanishing gradient problem that results in the starvation of many capsules
during training. | Source: | arXiv, 2301.01583 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|