|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'928
Articles rated: 2609
25 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Imitation networks: Few-shot learning of neural networks from scratch | | Akisato Kimura
; Zoubin Ghahramani
; Koh Takeuchi
; Tomoharu Iwata
; Naonori Ueda
; | | Date: |
8 Feb 2018 | | Abstract: | In this paper, we propose imitation networks, a simple but effective method
for training neural networks with a limited amount of training data. Our
approach inherits the idea of knowledge distillation that transfers knowledge
from a deep or wide reference model to a shallow or narrow target model. The
proposed method employs this idea to mimic predictions of reference estimators
that are much more robust against overfitting than the network we want to
train. Different from almost all the previous work for knowledge distillation
that requires a large amount of labeled training data, the proposed method
requires only a small amount of training data. Instead, we introduce pseudo
training examples that are optimized as a part of model parameters.
Experimental results for several benchmark datasets demonstrate that the
proposed method outperformed all the other baselines, such as naive training of
the target model and standard knowledge distillation. | | Source: | arXiv, 1802.3039 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER