| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
08 February 2025 |
|
| | | |
|
Article overview
| |
|
WLD-Reg: A Data-dependent Within-layer Diversity Regularizer | Firas Laakom
; Jenni Raitoharju
; Alexandros Iosifidis
; Moncef Gabbouj
; | Date: |
3 Jan 2023 | Abstract: | Neural networks are composed of multiple layers arranged in a hierarchical
structure jointly trained with a gradient-based optimization, where the errors
are back-propagated from the last layer back to the first one. At each
optimization step, neurons at a given layer receive feedback from neurons
belonging to higher layers of the hierarchy. In this paper, we propose to
complement this traditional ’between-layer’ feedback with additional
’within-layer’ feedback to encourage the diversity of the activations within
the same layer. To this end, we measure the pairwise similarity between the
outputs of the neurons and use it to model the layer’s overall diversity. We
present an extensive empirical study confirming that the proposed approach
enhances the performance of several state-of-the-art neural network models in
multiple tasks. The code is publically available at
url{this https URL} | Source: | arXiv, 2301.01352 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|