|
| | | | |
| | | |
| Stat |
Members: 3644
Articles: 2'497'992
Articles rated: 2609
16 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Learning Compact Recurrent Neural Networks | | Zhiyun Lu
; Vikas Sindhwani
; Tara N. Sainath
; | | Date: |
9 Apr 2016 | | Abstract: | Recurrent neural networks (RNNs), including long short-term memory (LSTM)
RNNs, have produced state-of-the-art results on a variety of speech recognition
tasks. However, these models are often too large in size for deployment on
mobile devices with memory and latency constraints. In this work, we study
mechanisms for learning compact RNNs and LSTMs via low-rank factorizations and
parameter sharing schemes. Our goal is to investigate redundancies in recurrent
architectures where compression can be admitted without losing performance. A
hybrid strategy of using structured matrices in the bottom layers and shared
low-rank factors on the top layers is found to be particularly effective,
reducing the parameters of a standard LSTM by 75%, at a small cost of 0.3%
increase in WER, on a 2,000-hr English Voice Search task. | | Source: | arXiv, 1604.2594 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER