| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
08 February 2025 |
|
| | | |
|
Article overview
| |
|
Some Insights into the Geometry and Training of Neural Networks | Ewout van den Berg
; | Date: |
2 May 2016 | Abstract: | Neural networks have been successfully used for classification tasks in a
rapidly growing number of practical applications. Despite their popularity and
widespread use, there are still many aspects of training and classification
that are not well understood. In this paper we aim to provide some new insights
into training and classification by analyzing neural networks from a
feature-space perspective. We review and explain the formation of decision
regions and study some of their combinatorial aspects. We place a particular
emphasis on the connections between the neural network weight and bias terms
and properties of decision boundaries and other regions that exhibit varying
levels of classification confidence. We show how the error backpropagates in
these regions and emphasize the important role they have in the formation of
gradients. These findings expose the connections between scaling of the weight
parameters and the density of the training samples. This sheds more light on
the vanishing gradient problem, explains the need for regularization, and
suggests an approach for subsampling training data to improve performance. | Source: | arXiv, 1605.0329 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|