|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'585
Articles rated: 2609
24 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Continual Learning with Self-Organizing Maps | | Pouya Bashivan
; Martin Schrimpf
; Robert Ajemian
; Irina Rish
; Matthew Riemer
; Yuhai Tu
; | | Date: |
19 Apr 2019 | | Abstract: | Despite remarkable successes achieved by modern neural networks in a wide
range of applications, these networks perform best in domain-specific
stationary environments where they are trained only once on large-scale
controlled data repositories. When exposed to non-stationary learning
environments, current neural networks tend to forget what they had previously
learned, a phenomena known as catastrophic forgetting. Most previous approaches
to this problem rely on memory replay buffers which store samples from
previously learned tasks, and use them to regularize the learning on new ones.
This approach suffers from the important disadvantage of not scaling well to
real-life problems in which the memory requirements become enormous. We propose
a memoryless method that combines standard supervised neural networks with
self-organizing maps to solve the continual learning problem. The role of the
self-organizing map is to adaptively cluster the inputs into appropriate task
contexts - without explicit labels - and allocate network resources
accordingly. Thus, it selectively routes the inputs in accord with previous
experience, ensuring that past learning is maintained and does not interfere
with current learning. Out method is intuitive, memoryless, and performs on par
with current state-of-the-art approaches on standard benchmarks. | | Source: | arXiv, 1904.9330 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER